Re: REST API notes
by Barry Warsaw
On Jun 28, 2017, at 01:47 AM, tlhackque--- via Mailman-users wrote:
>As a beginner with the REST API, I found it necessary to read code and
>experiment to figure out how to get things done.
Thanks for the feedback. Other than the work supporting Postorius and
HyperKitty, I think you've done some of the more extensive exploration of the
REST API, so your comments here (and greatly appreciated bug reports) are
quite valuable, and I think will help us make the REST API both more usable
and more discoverable.
It's important to point out some history of the REST API. Originally, we used
the restish library (which was Python 2 and eventually abandoned upstream).
That choice informed the essentially dynamic nature of the REST API. Which
means that unlike many other systems, we don't have -nor do I think we could
if we wanted to- easily and automatically generate a static description of the
REST API.
When I switched to Falcon (which is both Python 3 compatible and well
maintained upstream), I had to reimplement what I called "object based
traversal". Falcon by default supports a routes based traversal mechanism,
similar to Flask and other systems. But by plugging in (originally, by
monkeypatching, but now through an official Falcon API) support for object
based traversal, I was able to port the extensive existing REST implementation
to the new underlying library fairly easily.
All this to say that some of the problems you describe are due to historical
accident. And that's served us well(-ish) while we only had to support
mailmanclient and P/HK. But as the REST API is used in more places, those
deficiencies are becoming more glaring. I think if we ever make progress on
Lemme (our authenticating REST proxy), we'll likely encounter the same
problems. But at this point, it's infeasible to completely reimplement the
REST machinery.
>There are a number of implementation choices that are unusual.
No doubt. :)
>There is some documentation in src/mailman/rest/docs. It's oriented toward
>Python and intermixes the mailman client & REST API, apparently trying to
>show equivalence.
Much of that is because these are doctests, i.e. testable documentation. It's
a tradeoff between making them useful as docs but also testable without too
much clutter. It's very much oriented toward Python because that makes
testability easier. It would be good to have documentation for pure HTTP/JSON
consumers, but it would be imperative for that also to be testable so we're
sure it remains accurate, aside from any requirements to keep the docs in
sync. Suggestions and contributions would be very much welcome here.
>The REST API has a hybrid interface: Requests are made with
>application/x-www-form-urlencoded POST, PUT, PATCH and delete http requests.
>Requests are also accepted as parameter strings.
>
>The responses are JSON. (This is rather surprising - one would expect JSON
>requests - and I hope someday they'll be accepted, as the split complicates
>clients. I suspect this evolved to simplify the initial Web GUI client
>(Postorious), but it precludes using standard JSON-in/JSON-out client
>libraries.)
Absolutely. I want to support JSON-in/JSON-out. Again, the current mismatch
is due to historical implementation decisions, but there's nothing in
principle preventing us from accepting JSON in, afaict. We'd have to continue
to accept both, for backward compatibility reasons.
>Error responses are often misidentified as content-type: application/json,
>but actually contain a text/plain error message. This isn't universally
>true; for example the 401 response actually IS JSON. A client has to guess
>and handle decoding exceptions.
That's clearly a bug. The Content-Type should always be accurate. It would
be a new feature to also support JSON error responses.
>The API presents resources hierarchically, rooted at '/'. The top level
>resources include /users, /lists. /addresses, /domains, /system
>
>The next level is a resource id.
Kind of. It all depends on what the top-level resource is returned. This
makes sense when you think about the object based traversal machinery.
Each path component points internally to an object, and the subsequent path
component is handled by that object. If that returns another object, then the
next remaining path component is consumed by *that* object. And so on until
we reach the end of the path. Then the object at the leaf responds to the
HTTP command, and each object knows how to represent itself as JSON, and it
knows its canonical location within the resource tree (there can be multiple
paths to any particular object, but there's always one canonical location).
>There is a an id which is stable for the lifetime of an object, and a current
>name (which can be changed). For example, a list has a name like
>mylist(a)example.com, and a list_id of mylist.example.com. But if you change
>the list name, it becomes mynewname(a)example.net, while the list_id remains
>mylist.example.com.
Lists are weird in that they have two identifiers. One is the posting
address, what we internally (and slightly incorrectly) call the "fully
qualified list name". I don't particularly like that nomenclature anymore,
but we live with it.
A posting address can change, e.g. if you rename or rehome a list.
The (RFC 2919) List-ID is assigned when the mailing list is created and it's
immutable. Section 4 discourages changing the List-ID and Mailman takes that
as a requirement. Rename a list or rehome it to a different domain on the
same server and the List-ID will never change.
Again for historical reasons, many APIs both internal and external used the
posting address to identify the list, but this is wrong exactly because that
can change. I've slowly been converting APIs to accept both the posting
address and the List-ID when identifying the mailing list. New APIs generally
accept only the List-ID. Bottom line, it's best to use the List-ID.
>Lists are associated with "domains", which are the "domain part" of the
>list's address. That is, the part after the @. This is sometimes referred
>to as the "mail host", but there need not be a real host.
Yep, again historical nomenclature.
>A domain has to be created before you can create a list with an address in
>that domain.
Some APIs (e.g. `mailman create` CLI) will create the domain automatically if
it doesn't already exist, and unless you disable that explicitly.
>Resources are created with a POST to their top-level resource. To create a
>domain, post to /domains with mail_host => the domain, and (optionally)
>description => a description for the GUI. The response isn't JSON as one
>would expect. In fact, it's an empty application/json response with a 201
>status.
>
>To create a list, one POSTs to /lists. This post takes a restricted set of
>parameters; in the case of a list, just its fqdn_listname, (and an optional
>style - which isn't well defined). The response isn't JSON as one would
>expect. The Location header of the response contains a URL of the new list.
What else would you expect? From my reading of books like RESTful Web
Services (admittedly a long while ago), that's exactly the proper response to
an appending POST. Return a 201 with a Location header to the new resource
and empty content.
>To configure the list, you have to follow up with a PUT or PATCH to that URL
>+ /config This is where you can set the description, posting policy, etc.
>It's unrealistic to do a PUT; even if you're cloning another list, there's no
>programatic way to determine which attributes are writable. PATCH what you
>know...
Mailing list resources are somewhat unique in that they probably have the most
properties of any resource/object in the system. That's not surprising if you
think about it, but it does make PUTing to a mailing list more or less
impractical. That's certainly not true of other, smaller resources though,
and of course, you still want symmetry there (plus, implementation-wise, it's
almost a no brainer to support both PUT and PATCH).
>Members are associated with e-mail addresses - which belong to Users.
Users can have multiple addresses, addresses can be linked to only one user,
but may be unlinked, and members associate "subscribers" to mailing lists,
where a subscriber is either a user-with-preferred-address, or an address.
>You create a user by posting to /users with email => the email address, and
>optionally display_name => the name string. (A user can also be implicitly
>created by subscribing an e-mail address, but that gets confusing.) The
>e-mail address is the primary email address for the user. More later.
>Again, you get a Location header in the response, which you can use to PATCH
>/preferences to set delivery_status, etc. .These preferences are part of a
>hierarchy - many have a system default, a list default, the user default, and
>a subscription to that list value. You can find the User associated with a
>list by a GET of /addresses/address(a)example.net.
Technically, the returned user isn't necessarily associated with a list. It
*may* be subscribed to the list, but that relationship is represented by a
member.
>This GET returns two URLs: user => the user owning this address, and
>self_link => the address object.
*All* resources in the REST API have a self_link, and while that may seem
redundant, it's not. As mentioned above, you can take various paths through
the resource tree to get to a particular resource, but regardless of that,
every resource has exactly one canonical location. That location is
represented by the self_link.
>Note also that there are both an email and original_email attribute. The
>latter preserves case. The former is used internally by Mailman as a
>resource key, etc. (Though exactly what happens if John(a)example.net and
>john(a)example.net both subscribe is unclear.)
They can't. Mailman is case-preserving case-insensitive for email addresses.
Technically speaking, john(a)example.com and JOHN(a)example.com can be different
mailboxes, but that never happens in practice anymore, and Mailman has always
explicitly treated them as the same address. This goes back to the earliest
days of Mailman.
>Once a list is configured, you can add members. This requires the list_id -
>which you don't (officially) have. So you do a GET on the list resource, to
>get the list_id. Then you can subscribe a member with list_id => (the list
>id), subscriber => email address. Optionally you can
>pre_verify/confirm/approve the member and/or add a display_name.. POST to
>/members. Again, you get a Location header back. You can't specify
>everything you might like. as a creation attribute. So you may have to PATCH
>the member to, for example, set the moderation_status.
This is true, but it also kind of makes sense if you grok the way preferences
work, i.e. hierarchically as you describe above. TBH, I don't particularly
like the way preferences are modeled either internally or through the REST
API, but I haven't been able to come up with anything better.
>You may also need to PATCH the member /preferences if you want to set
>list-level delivery status, etc.
All true. In each of those cases, you may be talking to a different
preference resource.
>Consider adding at least an owner when creating a list.
I think that would be a useful addition to the REST API for list creation.
>One challenge is that almost everything requires multiple REST operations to
>set up. But REST (by definition) is stateless. So the best you can do is
>order operations & hope.
I don't understand this. REST is stateless, but not all HTTP operations are
idempotent. POST certainly isn't, by definition, so if you use it to create a
new resource under a collection, you clearly cannot modify that resource
through PUT or PATCH until the resource is created, which only happens when
POST succeeds.
>The mailman client examples refer to transactions (e.g in users.rst, there is
>'transaction.commit(); - but REST can't hold state. It does appear that the
>server uses DB transactions to ensure that any given REST operation is ACID,
>but the composite operations (e.g. create a list and set it's config) can not
>be Atomic. This is an architectural flaw in the API.
I think what you're getting at is that some POSTs to create new resources do
not accept the same parameters as the subsequent PUT or PATCH on the newly
created resource. I think that may be true, but it's not universally so.
E.g. POST on /domain creates a new domain, returning the location of the new
resource. You can provide both a description and an owner, and you can also
provide a mail_host. But mail_host is immutable, and PATCH or PUT support
changing all mutable attributes on the domain.
That should be the general rule; if an attribute on a resource is mutable, you
should be able to PATCH and PUT it, *and* you should be able to specify all
mutable attributes, plus some immutable ones on the POST that creates the
resource. GET should return all attributes, mutable or immutable. This rule
may not be strictly adhered to for all resources and collections, but I would
consider that a bug, not an architectural flaw.
AFAIK, this is all working as any well-defined web service should work.
>Symbolic names (which are required) for attributes are in the
>src/mailman/interfaces/(class).py.
Please note that the interfaces under this package are for the *internal* API,
which often, but not always, is exposed in the REST API. These two APIs serve
different purposes so they cannot be a one-to-one mapping, and there are many
resources in the REST API that don't correspond directly to internal objects,
and there are many internal objects that are not exposed to the REST API.
>What you can get/set comes from src/mailman/rest/(class).py.
Yes, so clients of the REST API are served by objects in this package.
Plugins and other internal operations are served by the objects that implement
interfaces in the src/mailman/interfaces package. This is a strict separation
of concerns, and it mirrors other 'external facing' interfaces, of which the
command line is another example.
Hope that helps.
-Barry
6 years, 10 months
Re: Migrating mailman3 to latest ubuntu lts
by Helio Loureiro
And using the module manually works...
(venv) mailman@new-server ~ (v3.3.9) [0|127]> *ipython3*
/usr/lib/python3/dist-packages/IPython/core/interactiveshell.py:949:
UserWarning: Attempting to work in a virtualenv. If you encounter problems,
please install IPython inside the virtualenv.
warn(
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: *import MySQLdb*
(venv) mailman@new-sever ~ (v3.3.9)> *fg*
Send job 1, 'ipython3' to foreground
In [3]: *con = MySQLdb.connect("localhost", "mailman3web",
"*****************", "mailman3web")*
In [4]:
*cursor = con.cursor()*
In [5]: *cursor.execute("show tables")*
Out[5]: 34
In [6]: *print(cursor.fetchall())*
(('account_emailaddress',), ('account_emailconfirmation',),
('auth_group',), ('auth_group_permissions',), ('auth_permission',),
('auth_user',), ('auth_user_groups',), ('auth_user_user_permissions',),
('django_admin_log',), ('django_content_type',),
('django_mailman3_maildomain',), ('django_mailman3_profile',),
('django_migrations',), ('django_q_ormq',), ('django_q_schedule',),
('django_q_task',), ('django_session',), ('django_site',),
('hyperkitty_attachment',), ('hyperkitty_email',),
('hyperkitty_favorite',), ('hyperkitty_lastview',),
('hyperkitty_mailinglist',), ('hyperkitty_profile',),
('hyperkitty_sender',), ('hyperkitty_tag',), ('hyperkitty_tagging',),
('hyperkitty_thread',), ('hyperkitty_threadcategory',),
('hyperkitty_vote',), ('socialaccount_socialaccount',),
('socialaccount_socialapp',), ('socialaccount_socialapp_sites',),
('socialaccount_socialtoken',))
In [7]: *cursor.execute("select * from account_emailaddress")*
Out[7]: 2530
In [8]: *print(cursor.fetchall())*
So what can I do? Is there a way to troubleshoot the root cause of this
issue?
Best Regards,
Helio Loureiro
https://helio.loureiro.eng.br
https://github.com/helioloureiro
https://mastodon.social/@helioloureiro
On Tue, 19 Dec 2023 at 11:11, Helio Loureiro <helio(a)loureiro.eng.br> wrote:
> Hi,
>
> No luck :(
>
> (venv) mailman@new-server ~ (v3.3.9)> *pip freeze | egrep
> "mailman-web|django-mailman3|django-allauth"*
> django-allauth==0.59.0
> django-mailman3==1.3.11
> mailman-web==0.0.8
> (venv) mailman@new-server ~ (v3.3.9)> *pip install -U
> django-allauth==0.58.0*
> Collecting django-allauth==0.58.0
> Downloading django-allauth-0.58.0.tar.gz (861 kB)
> ---------------------------------------- 861.7/861.7 KB 9.4 MB/s eta
> 0:00:00
> Installing build dependencies ... done
> Getting requirements to build wheel ... done
> Installing backend dependencies ... done
> Preparing metadata (pyproject.toml) ... done
> Requirement already satisfied: requests-oauthlib>=0.3.0 in
> ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (1.3.1)
> Requirement already satisfied: Django>=3.2 in
> ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (4.1.13)
> Requirement already satisfied: pyjwt[crypto]>=1.7 in
> ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (2.8.0)
> Requirement already satisfied: requests>=2.0.0 in
> ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (2.31.0)
> Requirement already satisfied: python3-openid>=3.0.8 in
> ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (3.2.0)
> Requirement already satisfied: asgiref<4,>=3.5.2 in
> ./venv/lib/python3.10/site-packages (from
> Django>=3.2->django-allauth==0.58.0) (3.7.2)
> Requirement already satisfied: sqlparse>=0.2.2 in
> ./venv/lib/python3.10/site-packages (from
> Django>=3.2->django-allauth==0.58.0) (0.4.4)
> Requirement already satisfied: cryptography>=3.4.0 in
> ./venv/lib/python3.10/site-packages (from
> pyjwt[crypto]>=1.7->django-allauth==0.58.0) (41.0.7)
> Requirement already satisfied: defusedxml in
> ./venv/lib/python3.10/site-packages (from
> python3-openid>=3.0.8->django-allauth==0.58.0) (0.7.1)
> Requirement already satisfied: urllib3<3,>=1.21.1 in
> ./venv/lib/python3.10/site-packages (from
> requests>=2.0.0->django-allauth==0.58.0) (2.1.0)
> Requirement already satisfied: charset-normalizer<4,>=2 in
> ./venv/lib/python3.10/site-packages (from
> requests>=2.0.0->django-allauth==0.58.0) (3.3.2)
> Requirement already satisfied: idna<4,>=2.5 in
> ./venv/lib/python3.10/site-packages (from
> requests>=2.0.0->django-allauth==0.58.0) (3.6)
> Requirement already satisfied: certifi>=2017.4.17 in
> ./venv/lib/python3.10/site-packages (from
> requests>=2.0.0->django-allauth==0.58.0) (2023.11.17)
> Requirement already satisfied: oauthlib>=3.0.0 in
> ./venv/lib/python3.10/site-packages (from
> requests-oauthlib>=0.3.0->django-allauth==0.58.0) (3.2.2)
> Requirement already satisfied: typing-extensions>=4 in
> ./venv/lib/python3.10/site-packages (from
> asgiref<4,>=3.5.2->Django>=3.2->django-allauth==0.58.0) (4.9.0)
> Requirement already satisfied: cffi>=1.12 in
> ./venv/lib/python3.10/site-packages (from
> cryptography>=3.4.0->pyjwt[crypto]>=1.7->django-allauth==0.58.0) (1.16.0)
> Requirement already satisfied: pycparser in
> ./venv/lib/python3.10/site-packages (from
> cffi>=1.12->cryptography>=3.4.0->pyjwt[crypto]>=1.7->django-allauth==0.58.0)
> (2.21)
> Building wheels for collected packages: django-allauth
> Building wheel for django-allauth (pyproject.toml) ... done
> Created wheel for django-allauth:
> filename=django_allauth-0.58.0-py3-none-any.whl size=1157319
> sha256=a430c552101d1ad47bc00b16d1c1d6df728afacdd13823927b4cbfb02c35dbfc
> Stored in directory:
> /local/mailman/.cache-ubuntu-22.04/pip/wheels/55/0a/79/e199827a18f310906c2a90b0e92b89c41daf21d2a502db6710
> Successfully built django-allauth
> Installing collected packages: django-allauth
> Attempting uninstall: django-allauth
> Found existing installation: django-allauth 0.59.0
> Uninstalling django-allauth-0.59.0:
> Successfully uninstalled django-allauth-0.59.0
> Successfully installed django-allauth-0.58.0
> (venv) mailman@new-server ~ (v3.3.9)> *mailman-web migrate*
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you don't
> care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't care
> about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying account.0004_alter_emailaddress_drop_unique_email...Traceback
> (most recent call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 89, in _execute
> return self.cursor.execute(sql, params)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
> line 75, in execute
> return self.cursor.execute(query, args)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 179, in execute
> res = self._query(mogrified_query)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 330, in _query
> db.query(q)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
> line 257, in query
> _mysql.connection.query(self, query)
> MySQLdb.OperationalError: (2013, 'Lost connection to MySQL server during
> query')
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File "/local/mailman/venv/bin/mailman-web", line 8, in <module>
> sys.exit(main())
> File
> "/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py",
> line 90, in main
> execute_from_command_line(sys.argv)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
> line 446, in execute_from_command_line
> utility.execute()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
> line 440, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
> line 402, in run_from_argv
> self.execute(*args, **cmd_options)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
> line 448, in execute
> output = self.handle(*args, **options)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
> line 96, in wrapped
> res = handle_func(*args, **kwargs)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py",
> line 349, in handle
> post_migrate_state = executor.migrate(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
> line 135, in migrate
> state = self._migrate_all_forwards(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
> line 167, in _migrate_all_forwards
> state = self.apply_migration(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
> line 252, in apply_migration
> state = migration.apply(state, schema_editor)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/migration.py",
> line 130, in apply
> operation.database_forwards(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/operations/fields.py",
> line 235, in database_forwards
> schema_editor.alter_field(from_model, from_field, to_field)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
> line 788, in alter_field
> self._alter_field(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
> line 858, in _alter_field
> self.execute(self._delete_unique_sql(model, constraint_name))
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
> line 199, in execute
> cursor.execute(sql, params)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 67, in execute
> return self._execute_with_wrappers(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 80, in _execute_with_wrappers
> return executor(sql, params, many, context)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 84, in _execute
> with self.db.wrap_database_errors:
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/utils.py", line
> 91, in __exit__
> raise dj_exc_value.with_traceback(traceback) from exc_value
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 89, in _execute
> return self.cursor.execute(sql, params)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
> line 75, in execute
> return self.cursor.execute(query, args)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 179, in execute
> res = self._query(mogrified_query)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 330, in _query
> db.query(q)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
> line 257, in query
> _mysql.connection.query(self, query)
> django.db.utils.OperationalError: (2013, 'Lost connection to MySQL server
> during query')
> (venv) mailman@new-server ~ (v3.3.9) [0|1]> *more
> /etc/mailman3/settings.py*
> # Mailman Web configuration file.
> # /etc/mailman3/settings.py
>
> # Get the default settings.
> from mailman_web.settings.base import *
> from mailman_web.settings.mailman import *
>
> # Settings below supplement or override the defaults.
>
> #: Default list of admins who receive the emails from error logging.
> ADMINS = (
> ('Mailman Suite Admin', 'root@localhost'),
> )
>
> # Postgresql database setup.
> DATABASES = {
> 'default': {
> 'ENGINE': 'django.db.backends.mysql',
> 'NAME': 'mailman3web',
> 'USER': 'mailman3web',
> # TODO: Replace this with the password.
> 'PASSWORD': '***********',
> 'HOST': 'localhost',
> # PORT: set to empty string for default.
> 'PORT': '3306',
> # OPTIONS: Extra parameters to use when connecting to the database.
> #'OPTIONS': {
> # Set sql_mode to 'STRICT_TRANS_TABLES' for MySQL. See
> # https://docs.djangoproject.com/en/1.11/ref/
> # databases/#setting-sql-mode
> # 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
> # 'charset': 'utf8mb4',
> #},
>
> }
> }
>
> # 'collectstatic' command will copy all the static files here.
> # Alias this location from your webserver to `/static`
> STATIC_ROOT = '/local/mailman/web/static'
>
> # enable the 'compress' command.
> COMPRESS_ENABLED = True
>
> # Make sure that this directory is created or Django will fail on start.
> LOGGING['handlers']['file']['filename'] =
> '/local/mailman/web/logs/mailmanweb.log'
>
> #: See https://docs.djangoproject.com/en/dev/ref/settings/#allowed-hosts
> ALLOWED_HOSTS = [
> "localhost", # Archiving API from Mailman, keep it.
> "127.0.0.1",
> # "lists.your-domain.org",
> # Add here all production domains you have.
> "*"
> ]
>
> #: See
> https://docs.djangoproject.com/en/dev/ref/settings/#csrf-trusted-origins
> (venv) mailman@new-server ~ (v3.3.9)> *mysql -umailman3web -p -h
> localhost mailman3web*
> Enter password:
> Reading table information for completion of table and column names
> You can turn off this feature to get a quicker startup with -A
>
> Welcome to the MariaDB monitor. Commands end with ; or \g.
> Your MariaDB connection id is 32
> Server version: 10.6.12-MariaDB-0ubuntu0.22.04.1 Ubuntu 22.04
>
> Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
>
> Type 'help;' or '\h' for help. Type '\c' to clear the current input
> statement.
>
> MariaDB [mailman3web]> show tables;
> +-------------------------------+
> | Tables_in_mailman3web |
> +-------------------------------+
> | account_emailaddress |
> | account_emailconfirmation |
> | auth_group |
> | auth_group_permissions |
> | auth_permission |
> | auth_user |
> | auth_user_groups |
> | auth_user_user_permissions |
> | django_admin_log |
> | django_content_type |
> | django_mailman3_maildomain |
> | django_mailman3_profile |
> | django_migrations |
> | django_q_ormq |
> | django_q_schedule |
> | django_q_task |
> | django_session |
> | django_site |
> | hyperkitty_attachment |
> | hyperkitty_email |
> | hyperkitty_favorite |
> | hyperkitty_lastview |
> | hyperkitty_mailinglist |
> | hyperkitty_profile |
> | hyperkitty_sender |
> | hyperkitty_tag |
> | hyperkitty_tagging |
> | hyperkitty_thread |
> | hyperkitty_threadcategory |
> | hyperkitty_vote |
> | socialaccount_socialaccount |
> | socialaccount_socialapp |
> | socialaccount_socialapp_sites |
> | socialaccount_socialtoken |
> +-------------------------------+
> 34 rows in set (0.000 sec)
>
> Best Regards,
> Helio Loureiro
> https://helio.loureiro.eng.br
> https://github.com/helioloureiro
> https://mastodon.social/@helioloureiro
>
>
> On Mon, 18 Dec 2023 at 17:11, Mark Sapiro <mark(a)msapiro.net> wrote:
>
>> On 12/18/23 6:24 AM, Helio Loureiro wrote:
>> > Hi,
>> >
>> > Indeed it was the configuration. It was placed into
>> > /etc/mailman3/mailman-web.py. After a I changed to
>> > /etc/mailman3/settings.py a few things advanced a little bit more.
>> >
>> > I had to figure out how to fix mysqlclient installation since there
>> isn't a
>> > mention about it and the simple "pip install mysqclient" was breaking
>> with
>> > pkg-config issues. But it did work at the end.
>> >
>> > Now I can see further messages on mailman3-web than before.
>> >
>> > (venv) mailman@new-server ~ (v3.3.9)> mailman-web migrate
>> > System check identified some issues:
>> >
>> > WARNINGS:
>> > account.EmailAddress: (models.W036) MariaDB does not support unique
>> > constraints with conditions.
>> > HINT: A constraint won't be created. Silence this warning if you don't
>> care
>> > about it.
>> > account.EmailAddress: (models.W043) MariaDB does not support indexes on
>> > expressions.
>> > HINT: An index won't be created. Silence this warning if you don't care
>> > about it.
>> > Operations to perform:
>> > Apply all migrations: account, admin, auth, contenttypes,
>> > django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
>> > socialaccount
>> > Running migrations:
>> > Applying
>> account.0004_alter_emailaddress_drop_unique_email...Traceback
>> > (most recent call last):
>>
>>
>> I'm not sure why there would be an issue with this migration, but there
>> is a possible compatibility issue depending on how you installed things.
>>
>> django-mailman3<=1.3.11 is not compatible with django-allauth>=0.58.
>>
>> In your venv, try
>> ```
>> pip install django-allauth\<0.58
>> ```
>>
>> --
>> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
>> San Francisco Bay Area, California better use your sense - B. Dylan
>>
>> _______________________________________________
>> Mailman-users mailing list -- mailman-users(a)mailman3.org
>> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>> Archived at:
>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>>
>> This message sent to helio(a)loureiro.eng.br
>>
>
5 months
Mailman3 in dockers on EC2 - running out of memory and puking ev.ry.where.
by roger.hislop＠is.co.za
Hi all -- really need some help. I'm tech literate, but definitely not a sysadmin or dev.
I'm following the tutorial at "Olay's Farmland" at https://www.olay.xyz/2018/01/01/deploy-mailman-3-on-aws-using-docker/ which seems to be a clone of the same content here https://xiaoxing.us/2018/01/01/deploy-mailman-3-on-aws-using-docker/
Easy enough to get most of the way ... set up EC2 instance, configure SES and add DNS settings with DKIM etc. Set up Postfix, test it. All working groovy.
I run through the how to all the way through setting up the containers and firing it up. This is where things start to get funky.
Once the docker-compose is run, I run "curl http://172.19.199.3:8000/postorius/lists/" to test. First time it said it couldn't connect to database. So I nuked and restarted from scratch. Second time it said the same thing... was reading around the place and tried again, and then reran that curl and it worked -- gave me html from a page. But the VM started running slowly --taking forever to follow keypresses.
Eventually became unresponsive. So I rebooted the machine via AWS console, and docker ps showed only the postgres db running. docker-compose ps shows
Name Command State Ports
------------------------------------------------------------------------------------------
docker-mailman_database_1 docker-entrypoint.sh postgres Up 5432/tcp
mailman-core docker-entrypoint.sh maste ... Exit 255 8001/tcp, 8024/tcp
mailman-web docker-entrypoint.sh uwsgi ... Exit 255 8000/tcp, 8080/tcp
So rebuilt the container -- same thing. It looks and feels like its running out of memory.
I used the compose file from https://gist.githubusercontent.com/Yexiaoxing/833bfcc5d3e4e0c06a8b7f0bac7c4… (with appropriate edits).
I have no idea how to proceed from here...
Go to logs...
Partial log (they're LONG)
docker logs docker-mailman_database_1
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
sh: locale: not found
performing post-bootstrap initialization ... No usable system locales were found.
Use the option "--debug" to see details.
ok
syncing data to disk ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....LOG: database system was shut down at 2019-04-09 14:37:48 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
done
server started
CREATE DATABASE
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
LOG: received fast shutdown request
LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
waiting for server to shut down....LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
LOG: database system was shut down at 2019-04-09 14:37:50 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
ERROR: relation "social_auth_usersocialauth" does not exist at character 15
STATEMENT: SELECT 1 from social_auth_usersocialauth
LOG: database system was interrupted; last known up at 2019-04-09 14:48:37 UTC
LOG: database system was not properly shut down; automatic recovery in progress
LOG: invalid record length at 0/1767BB8: wanted 24, got 0
LOG: redo is not required
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
LOG: received smart shutdown request
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
LOG: database system was shut down at 2019-04-09 15:18:06 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
ERROR: duplicate key value violates unique constraint "auth_user_username_key"
DETAIL: Key (username)=(lists) already exists.
STATEMENT: INSERT INTO "auth_user" ("password", "last_login", "is_superuser", "username", "first_name", "last_name", "email", "is_staff", "is_active", "date_joined") VALUES ('!lS8pgTTV7CyhUbZItku00ZnUGxLQkVEuc3ENZKbV', NULL, true, 'lists', '', '', 'iotic-za(a)reasondigital.co.za', true, true, '2019-04-09T15:39:57.715036+00:00'::timestamptz) RETURNING "auth_user"."id"
TopMemoryContext: 8192 total in 1 blocks; 3480 free (0 chunks); 4712 used
smgr relation table: 24576 total in 2 blocks; 13008 free (4 chunks); 11568 used
WAL record construction: 49768 total in 2 blocks; 6584 free (0 chunks); 43184 used
Checkpointer: 8192 total in 1 blocks; 7992 free (3 chunks); 200 used
PrivateRefCount: 8192 total in 1 blocks; 2840 free (0 chunks); 5352 used
MdSmgr: 8192 total in 1 blocks; 8120 free (0 chunks); 72 used
Pending ops context: 0 total in 0 blocks; 0 free (0 chunks); 0 used
Pending Ops Table: 8192 total in 1 blocks; 2840 free (5 chunks); 5352 used
LOCALLOCK hash: 8192 total in 1 blocks; 776 free (0 chunks); 7416 used
Timezones: 104120 total in 2 blocks; 2840 free (0 chunks); 101280 used
ErrorContext: 8192 total in 1 blocks; 8152 free (0 chunks); 40 used
Grand total: 235808 bytes in 13 blocks; 56632 free (12 chunks); 179176 used
ERROR: out of memory
DETAIL: Failed on request of size 4032.
CONTEXT: writing block 0 of relation base/16384/16797
TopMemoryContext: 8192 total in 1 blocks; 3480 free (0 chunks); 4712 used
smgr relation table: 24576 total in 2 blocks; 13008 free (4 chunks); 11568 used
WAL record construction: 49768 total in 2 blocks; 6584 free (0 chunks); 43184 used
Checkpointer: 8192 total in 1 blocks; 7992 free (2 chunks); 200 used
PrivateRefCount: 8192 total in 1 blocks; 2840 free (0 chunks); 5352 used
LOG: could not fork autovacuum worker process: Out of memory
MdSmgr: 8192 total in 1 blocks; 8120 free (0 chunks); 72 used
Pending ops context: 0 total in 0 blocks; 0 free (0 chunks); 0 used
Pending Ops Table: 8192 total in 1 blocks; 2840 free (5 chunks); 5352 used
LOCALLOCK hash: 8192 total in 1 blocks; 776 free (0 chunks); 7416 used
LOG: could not fork autovacuum worker process: Out of memory
Timezones: 104120 total in 2 blocks; 2840 free (0 chunks); 101280 used
ErrorContext: 8192 total in 1 blocks; 8152 free (0 chunks); 40 used
Grand total: 235808 bytes in 13 blocks; 56632 free (11 chunks); 179176 used
LOG: could not fork autovacuum worker process: Out of memory
ERROR: out of memory
DETAIL: Failed on request of size 4032.
CONTEXT: writing block 0 of relation base/16384/16797
LOG: could not fork autovacuum worker process: Out of memory
WARNING: could not write block 0 of base/16384/16797
DETAIL: Multiple failures --- write error might be permanent.
TopMemoryContext: 8192 total in 1 blocks; 3480 free (0 chunks); 4712 used
smgr relation table: 24576 total in 2 blocks; 13008 free (4 chunks); 11568 used
WAL record construction: 49768 total in 2 blocks; 6584 free (0 chunks); 43184 used
Checkpointer: 8192 total in 1 blocks; 7992 free (2 chunks); 200 used
PrivateRefCount: 8192 total in 1 blocks; 2840 free (0 chunks); 5352 used
MdSmgr: 8192 total in 1 blocks; 8120 free (0 chunks); 72 used
Pending ops context: 0 total in 0 blocks; 0 free (0 chunks); 0 used
Pending Ops Table: 8192 total in 1 blocks; 2840 free (5 chunks); 5352 used
LOCALLOCK hash: 8192 total in 1 blocks; 776 free (0 chunks); 7416 used
TopMemoryContext: 51040 total in 4 blocks; 1344 free (1 chunks); 49696 used
TopTransactionContext: 8192 total in 1 blocks; 6968 free (0 chunks); 1224 used
Timezones: 104120 total in 2 blocks; 2840 free (0 chunks); 101280 used
Statistics snapshot: 0 total in 0 blocks; 0 free (0 chunks); 0 used
ErrorContext: 8192 total in 1 blocks; 8152 free (0 chunks); 40 used
Grand total: 235808 bytes in 13 blocks; 56632 free (11 chunks); 179176 used
Per-database function: 8192 total in 1 blocks; 776 free (0 chunks); 7416 used
Per-database table: 8192 total in 1 blocks; 776 free (0 chunks); 7416 used
Per-database function: 8192 total in 1 blocks; 776 free (0 chunks); 7416 used
Per-database table: 8192 total in 1 blocks; 776 free (0 chunks); 7416 used
Databases hash: 24576 total in 2 blocks; 15072 free (5 chunks); 9504 used
ERROR: out of memory
For UWSGI
cat uwsgi.log
[pid: 49|app: 0|req: 1/1] 172.19.199.1 () {24 vars in 291 bytes} [Tue Apr 9 14:43:43 2019] GET /postorius/lists/ => generated 3710 bytes in 705 msecs (HTTP/1.1 200) 5 headers in 163 bytes (1 switches on core 0)
[pid: 39|app: 0|req: 1/1] 172.19.199.1 () {24 vars in 291 bytes} [Tue Apr 9 15:40:12 2019] GET /postorius/lists/ => generated 3710 bytes in 694 msecs (HTTP/1.1 200) 5 headers in 163 bytes (1 switches on core 0)
[ec2-user@ip-172-31-23-250 logs]$ cat uwsgi-error.log
cat: uwsgi-error.log: Permission denied
[ec2-user@ip-172-31-23-250 logs]$ sudo cat uwsgi-error.log
*** Starting uWSGI 2.0.15 (64bit) on [Tue Apr 9 14:38:03 2019] ***
compiled with version: 6.3.0 on 29 December 2017 00:16:31
os: Linux-4.14.77-70.59.amzn1.x86_64 #1 SMP Mon Nov 12 22:02:45 UTC 2018
nodename: mailman-web
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /opt/mailman-web
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
setgid() to 101
setuid() to 100
chdir() to /opt/mailman-web
your processes number limit is 3860
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 0.0.0.0:8080 fd 8
uwsgi socket 1 bound to TCP address 0.0.0.0:8000 fd 9
Python version: 2.7.14 (default, Dec 19 2017, 17:52:21) [GCC 6.3.0]
Python main interpreter initialized at 0x55ca1d90e1e0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
[uwsgi-cron] command "./manage.py runjobs minutely" registered as cron task
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" registered as cron task
[uwsgi-cron] command "./manage.py runjobs hourly" registered as cron task
[uwsgi-cron] command "./manage.py runjobs daily" registered as cron task
[uwsgi-cron] command "./manage.py runjobs monthly" registered as cron task
[uwsgi-cron] command "./manage.py runjobs weekly" registered as cron task
[uwsgi-cron] command "./manage.py runjobs yearly" registered as cron task
mapped 166144 bytes (162 KB) for 2 cores
*** Operational MODE: threaded ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x55ca1d90e1e0 pid: 1 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1)
spawned uWSGI worker 1 (pid: 49, cores: 2)
Tue Apr 9 14:38:04 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 51)
[uwsgi-daemons] spawning "./manage.py qcluster" (uid: 100 gid: 101)
14:38:07 [Q] INFO Q Cluster-52 starting.
14:38:07 [Q] INFO Process-1 guarding cluster at 67
14:38:07 [Q] INFO Process-1:1 ready for work at 68
14:38:07 [Q] INFO Process-1:3 pushing tasks at 70
14:38:07 [Q] INFO Q Cluster-52 running.
14:38:07 [Q] INFO Process-1:2 monitoring at 69
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 51 exited after 4 second(s)
Tue Apr 9 14:39:04 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 71)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 71 exited after 269 second(s)
Tue Apr 9 14:43:33 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 79)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 79 exited after 3 second(s)
Tue Apr 9 14:51:35 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 89)
*** Starting uWSGI 2.0.15 (64bit) on [Tue Apr 9 15:39:58 2019] ***
compiled with version: 6.3.0 on 29 December 2017 00:16:31
os: Linux-4.14.109-80.92.amzn1.x86_64 #1 SMP Mon Apr 1 23:07:39 UTC 2019
nodename: mailman-web
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /opt/mailman-web
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
setgid() to 101
setuid() to 100
chdir() to /opt/mailman-web
your processes number limit is 3860
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 0.0.0.0:8080 fd 8
uwsgi socket 1 bound to TCP address 0.0.0.0:8000 fd 9
Python version: 2.7.14 (default, Dec 19 2017, 17:52:21) [GCC 6.3.0]
Python main interpreter initialized at 0x562e2c0b11e0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
[uwsgi-cron] command "./manage.py runjobs minutely" registered as cron task
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" registered as cron task
[uwsgi-cron] command "./manage.py runjobs hourly" registered as cron task
[uwsgi-cron] command "./manage.py runjobs daily" registered as cron task
[uwsgi-cron] command "./manage.py runjobs monthly" registered as cron task
[uwsgi-cron] command "./manage.py runjobs weekly" registered as cron task
[uwsgi-cron] command "./manage.py runjobs yearly" registered as cron task
mapped 166144 bytes (162 KB) for 2 cores
*** Operational MODE: threaded ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x562e2c0b11e0 pid: 1 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1)
spawned uWSGI worker 1 (pid: 39, cores: 2)
Tue Apr 9 15:39:59 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 41)
[uwsgi-daemons] spawning "./manage.py qcluster" (uid: 100 gid: 101)
15:40:02 [Q] INFO Q Cluster-43 starting.
15:40:02 [Q] INFO Process-1:1 ready for work at 58
15:40:02 [Q] INFO Process-1:2 monitoring at 59
15:40:02 [Q] INFO Process-1 guarding cluster at 57
15:40:02 [Q] INFO Process-1:3 pushing tasks at 60
15:40:02 [Q] INFO Q Cluster-43 running.
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 41 exited after 4 second(s)
Tue Apr 9 15:40:59 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 63)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 63 exited after 13 second(s)
Tue Apr 9 15:41:59 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 71)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 71 exited after 33 second(s)
Tue Apr 9 15:42:59 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 79)
16:33:36 [Q] ERROR could not translate host name "database" to address: Try again
16:35:17 [Q] INFO Process-1:3 stopped pushing tasks
16:35:25 [Q] ERROR reincarnated pusher Process-1:3 after sudden death
16:35:25 [Q] INFO Process-1:4 pushing tasks at 87
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 79 exited after 3146 second(s)
Tue Apr 9 16:35:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 88)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 88 exited after 3 second(s)
Tue Apr 9 16:36:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 96)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 96 exited after 3 second(s)
Tue Apr 9 16:37:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 104)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 104 exited after 3 second(s)
Tue Apr 9 16:38:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 112)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 112 exited after 3 second(s)
Tue Apr 9 16:39:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 120)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 120 exited after 3 second(s)
Tue Apr 9 16:40:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 128)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 128 exited after 3 second(s)
Tue Apr 9 16:41:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 136)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 136 exited after 3 second(s)
Tue Apr 9 16:42:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 144)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 144 exited after 3 second(s)
Tue Apr 9 16:43:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 152)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 152 exited after 3 second(s)
Tue Apr 9 16:44:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 160)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 160 exited after 3 second(s)
Tue Apr 9 16:45:00 2019 - [uwsgi-cron] running "./manage.py runjobs quarter_hourly" (pid 168)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running with pid 168 exited after 2 second(s)
Tue Apr 9 16:45:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 176)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 176 exited after 3 second(s)
Tue Apr 9 16:46:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 184)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 184 exited after 2 second(s)
Tue Apr 9 16:47:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 192)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 192 exited after 3 second(s)
Tue Apr 9 16:48:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 200)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 200 exited after 3 second(s)
Tue Apr 9 16:49:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 208)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 208 exited after 3 second(s)
Tue Apr 9 16:50:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 216)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 216 exited after 3 second(s)
Tue Apr 9 16:51:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 224)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 224 exited after 3 second(s)
Tue Apr 9 16:52:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 232)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 232 exited after 3 second(s)
Tue Apr 9 16:53:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 240)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 240 exited after 2 second(s)
Tue Apr 9 16:54:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 248)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 248 exited after 2 second(s)
Tue Apr 9 16:55:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 256)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 256 exited after 3 second(s)
Tue Apr 9 16:56:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 264)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 264 exited after 3 second(s)
Tue Apr 9 16:57:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 272)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 272 exited after 3 second(s)
Tue Apr 9 16:58:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 280)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 280 exited after 3 second(s)
Tue Apr 9 16:59:25 2019 - [uwsgi-cron] running "./manage.py runjobs minutely" (pid 288)
[uwsgi-cron] command "./manage.py runjobs minutely" running with pid 288 exited after 3 second(s)
Tue Apr 9 17:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs quarter_hourly" (pid 296)
Tue Apr 9 17:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs hourly" (pid 297)
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
utility.execute()
File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 337, in execute
utility.execute()
File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 337, in execute
django.setup()
django.setup()
File "/usr/local/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
File "/usr/local/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models()
app_config.import_models()
File "/usr/local/lib/python2.7/site-packages/django/apps/config.py", line 202, in import_models
File "/usr/local/lib/python2.7/site-packages/django/apps/config.py", line 202, in import_models
self.models_module = import_module(models_module_name)
self.models_module = import_module(models_module_name)
File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
__import__(name)
File "/usr/local/lib/python2.7/site-packages/hyperkitty/models/__init__.py", line 28, in <module>
File "/usr/local/lib/python2.7/site-packages/hyperkitty/models/__init__.py", line 28, in <module>
from .email import Email, Attachment
from .email import Email, Attachment
File "/usr/local/lib/python2.7/site-packages/hyperkitty/models/email.py", line 37, in <module>
File "/usr/local/lib/python2.7/site-packages/hyperkitty/models/email.py", line 37, in <module>
from hyperkitty.lib.analysis import compute_thread_order_and_depth
from hyperkitty.lib.analysis import compute_thread_order_and_depth
File "/usr/local/lib/python2.7/site-packages/hyperkitty/lib/analysis.py", line 28, in <module>
File "/usr/local/lib/python2.7/site-packages/hyperkitty/lib/analysis.py", line 28, in <module>
import networkx as nx
import networkx as nx
File "/usr/local/lib/python2.7/site-packages/networkx/__init__.py", line 114, in <module>
File "/usr/local/lib/python2.7/site-packages/networkx/__init__.py", line 114, in <module>
import networkx.generators
import networkx.generators
File "/usr/local/lib/python2.7/site-packages/networkx/generators/__init__.py", line 6, in <module>
File "/usr/local/lib/python2.7/site-packages/networkx/generators/__init__.py", line 6, in <module>
from networkx.generators.classic import *
from networkx.generators.classic import *
File "/usr/local/lib/python2.7/site-packages/networkx/generators/classic.py", line 26, in <module>
File "/usr/local/lib/python2.7/site-packages/networkx/generators/classic.py", line 26, in <module>
from networkx.algorithms.bipartite.generators import complete_bipartite_graph
from networkx.algorithms.bipartite.generators import complete_bipartite_graph
File "/usr/local/lib/python2.7/site-packages/networkx/algorithms/__init__.py", line 57, in <module>
File "/usr/local/lib/python2.7/site-packages/networkx/algorithms/__init__.py", line 57, in <module>
import networkx.algorithms.connectivity
import networkx.algorithms.connectivity
File "/usr/local/lib/python2.7/site-packages/networkx/algorithms/connectivity/__init__.py", line 3, in <module>
File "/usr/local/lib/python2.7/site-packages/networkx/algorithms/connectivity/__init__.py", line 3, in <module>
from .connectivity import *
from .connectivity import *
File "/usr/local/lib/python2.7/site-packages/networkx/algorithms/connectivity/connectivity.py", line 13, in <module>
File "/usr/local/lib/python2.7/site-packages/networkx/algorithms/connectivity/connectivity.py", line 13, in <module>
from networkx.algorithms.flow import boykov_kolmogorov
from networkx.algorithms.flow import boykov_kolmogorov
File "/usr/local/lib/python2.7/site-packages/networkx/algorithms/flow/__init__.py", line 1, in <module>
File "/usr/local/lib/python2.7/site-packages/networkx/algorithms/flow/__init__.py", line 1, in <module>
from .maxflow import *
from .maxflow import *
MemoryErrorMemoryError
18:16:37 [Q] ERROR could not translate host name "database" to address: Try again
...etc ...
For Mailman
cat mailmanweb.log
ERROR 2019-04-09 20:05:48,878 934 hyperkitty.lib.utils Failed to update the fulltext index: could not translate host name "database" to address: Try again
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/hyperkitty/lib/utils.py", line 186, in run_with_lock
fn(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/hyperkitty/search_indexes.py", line 87, in update_index
update_cmd.update_backend("hyperkitty", "default")
File "/usr/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 240, in update_backend
total = qs.count()
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 363, in count
return self.query.get_count(using=self.db)
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 498, in get_count
number = obj.get_aggregation(using, ['__count'])['__count']
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 479, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 874, in execute_sql
cursor = self.connection.cursor()
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 254, in cursor
return self._cursor()
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 229, in _cursor
self.ensure_connection()
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
self.connect()
File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
self.connect()
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 189, in connect
self.connection = self.get_new_connection(conn_params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/postgresql/base.py", line 176, in get_new_connection
connection = Database.connect(**conn_params)
File "/usr/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
OperationalError: could not translate host name "database" to address: Try again
ERROR 2019-04-09 20:15:08,192 983 hyperkitty.lib.utils Failed to update the fulltext index: could not translate host name "database" to address: Try again
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/hyperkitty/lib/utils.py", line 186, in run_with_lock
fn(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/hyperkitty/search_indexes.py", line 87, in update_index
update_cmd.update_backend("hyperkitty", "default")
File "/usr/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 240, in update_backend
total = qs.count()
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 363, in count
return self.query.get_count(using=self.db)
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 498, in get_count
number = obj.get_aggregation(using, ['__count'])['__count']
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 479, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 874, in execute_sql
cursor = self.connection.cursor()
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 254, in cursor
return self._cursor()
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 229, in _cursor
self.ensure_connection()
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
self.connect()
File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
self.connect()
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 189, in connect
self.connection = self.get_new_connection(conn_params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/postgresql/base.py", line 176, in get_new_connection
connection = Database.connect(**conn_params)
File "/usr/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
OperationalError: could not translate host name "database" to address: Try again
5 years, 1 month
Re: Importing mbox files into archive defect with lines with From
by Alex Schuilenburg
Hi Stephen
On 19/09/2023 19:59, Stephen J. Turnbull wrote:
> Did you intend to send this to me only?
Nope, apologies. I'm happy to add the list back in - I thought I hit
reply-to-list.
> Alex Schuilenburg writes:
>
> > Yes, I retested with from the git head as well. Same result. It
> > sees the line
> >
> > From 1st April 2019, our turnover has exceeded the threshold for
> >
> > as the start of a new message and reports no anomolies although
> > there is clearly one.
>
> If there is an empty line preceding that one, and the "F" is in the
> first column (no marginal whitespace), that is considered correct
> behavior for processing an mbox file.
As an email separator agreed, provided the body has all "First "
(preceded by a blank line and nothing preceding on the same line)
suitably escaped.
> If you don't like it, don't use the mbox format. The maildir
> message-per-file format is widely supported, and if you want a
> folder-per-file rather than message-per-file format, MMDF is fairly
> widely supported, and does not suffer from the ambiguities of mbox.
> All three (with variations) are supported by recent Python 3.
Understood. My issue is the lists were imported from MM2.1 in 2020 into
MM3.2.1 onto hyperkitty+mariadb/mysql.
So the current installation has no mbox files - archives are stored in a
mysql database.
> > The old MM3 installation I am migrating away from is obviously
> > broken in that its mbox archive when downloaded is not escaped,
>
> That's interesting. I presume that's because HyperKitty saves
> messages in a database rather than as mbox files, and the old mbox
> exporter just flattened and concatenated the messages. Or perhaps
> that also uses mailbox.mbox and old enough versions of that didn't
> From-stuff.
Spot on.
I have to move the lists onto a new Debian 12 server using the native
mailman 3.3.8 & mailman-web 0+20200530-2 packages. I tried dropping in
the old mailman3 database under the new software but that did not work.
Instead I manually imported the old mailman3 data directly into the new
mailman3 database as, after inspections, there were no new tables and
only a couple of additional fields which have suitable defaults. So
dumped the mailman3 data with
mysqldump --no-create-info --no-create-db --disable-keys
--complete-insert --ignore-table=mailman3.alembic_version mailman3
and simply imported the dump into the new schema. OK so far. The lists
showed up in postorius, obviously without archives in hyperkitty.
> The normal way to upgrade a HyperKitty archive is to do nothing, just
> upgrade the software. I guess you moved to a new host and deleted the
> database? The preferred way is to dump the database to SQL, and then
> load it in to the new database directly rather than downloading the
> mbox files and importing. No ambiguity and much faster.
Thats what I thought initially, but that failed as per
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message….
As my old installation appears to have the django_migrations table
inconsistent with the state of the database, and Debian have been
unresponsive so far.
Unfortunately the same mailman3-style manual import of mailman3web from
old mailman3web db into the new db was not possible. There were
additional tables and fields that needed values rather than defaults.
Instead I opted to download the mbox exports from the old installation
via the web interface and import them into the new installation for
expediency.This is where I was when I posted the message.
However, the old installation's mbox archive export (from the database)
was problematic (e.g. 5min timeout) but after getting around those
issues I ended up with some broken mbox files (e.g. the "From 1st April
..." line within a message, resulting in hyperkitty_import failures).
> > although I tested downloading this thread archive and it is clearly
> > escaped. Unfortunately I cannot tell if hyperkitty_import works
> > using the git head since I am restricted to using the version
> > provided with Debian 12.
>
> In July it worked for me on a site with 5300 archived lists. I assume
> some of them had From-stuffed lines. Can't be sure, most of the posts
> are machine-generated, but lines beginning with "From " are pretty
> common in natural English.
I'd be surprised if there were not any. I have 2003 lists and several
occurrences.
>
> > There are indeed two versions though of mailbox.py on the VM:
> >
> > /usr/lib/python3/dist-packages/mailman/utilities/mailbox.py: from
> > mailman3 (3.3.8-2~deb12u1)
> > /usr/lib/python3.11/mailbox.py from libpython3.11-stdlib (3.11.2-6)
>
> The first applies one small change to the second (a so-called
> "monkey-patch") for use by Mailman core. It should only be visible
> to Python by the name "mailman.utilities.mailbox". The HyperKitty
> utilities import the name "mailbox". If overwriting the first with
> the second changes HyperKitty's behavior, something is wrong with
> sys.path.
Then I guess that is the case in Debian 12.
>
> > the latter of which appears far more substantial , so I dropped
> > that module over the former and reran hyperkitty_import over the
> > '>' escaped mbox, and while it did import without error, it did
> > leave the escape in place (i.e. the 'From ...' was quoted when
> > viewed).
>
> This is considered correct behavior. It is not possible to determine
> whether the escape was in the original, or added by a receiving MTA.
> Better to leave it.
I thought that ">From " would be escaped to ">>From ", and so on, so the
escape could easily be reversed when imported. I tested exports from the
mailman-users lists and lines beginning "From " (preceded by a blank
line) are escaped to ">From ", so incorrectly figured this would be
unescaped by hyperkitty_import. After all, I would expect that an export
of the archive to mbox, followed by a delete of the archive, followed by
a hyperkitty_import of the archive, should leave you at the same place.
Not with ">From " escapes in the new archives. In fact I also had a
number of messages with "Message-ID: <>" and worse: all messages with
attachments had the text/plain content empty.
So mbox exports from MM 3.2.1 on Debian 10 (using hyperkitty+mysql) are
broken.
>
> > The unescaped mbox import died in the same way.
>
> As expected.
>
> > Anyhow, thanks for your suggestion. For now I can stick with a
> > manual repair and spaced escape of From_.
>
> If the old database is still available, I recommend dumping that and
> loading it into a fresh version of the DBMS.
> ...
Thanks for the pointer. As I had already done the same with mailman3,
so repeated the excercise. The following dump and import worked.
oldhost> mysqldump --no-create-info --no-create-db --disable-keys
--complete-insert mailman3web > mailman3web.sql
newhost> mysql
MariaDB [(none]> use mailman3web
MariaDB [mailman3web]> source mailman3web.sql
Thanks again
-- Alex
7 months, 4 weeks
Re: config incoming email in my domain cpanel
by tlhackque
On 03-Aug-17 17:14, Rafael Mora wrote:
>
>
> El jue., 3 ago. 2017 a las 16:07, tlhackque via Mailman-users
> (<mailman-users(a)mailman3.org <mailto:mailman-users@mailman3.org>>)
> escribió:
>
> You've said you're working with labbrands.com <http://labbrands.com>.
>
> labbrands.com <http://labbrands.com> has address 192.185.51.89
> mail.labbrands.com <http://mail.labbrands.com> has address
> 192.185.51.89
> www.labbrands.com <http://www.labbrands.com> is an alias for
> labbrands.com <http://labbrands.com>.
>
> telnet proves that Exim is running on that IP address. Either you're
> running it, or hostgator is redirecting the SMTP ports.
>
> You added:
> lists.labbrands.com <http://lists.labbrands.com> has address
> 190.145.27.66
>
> Presumably that's a different machine.
>
> So that now needs an MX record, and a firewall routing.
>
>
> I added the MX record on our hostgator hosting like this:
> image.png
>
> Is it correct???
>
No. And sending images to the mailing list doesn't get you help.
You set up lists.labbrands.com as a secondary server, not an independent
mail domain. This will cause you serious grief.
You need to get some help from a more suitable list. Setting this up is
not mailman-specific, and getting it right is more than a quick note.
Try the Postfix documentation & mailing lists.
>
>
> And the SPF
> records ... and spam and virus filters, and all the other stuff
> you need
> to setup an independent, public-facing smtp server.
>
> But at least you won't have to relay.
>
> Good luck.
>
> On 03-Aug-17 16:55, Rafael Mora wrote:
> > El jue., 3 ago. 2017 a las 15:49, tlhackque via Mailman-users (<
> > mailman-users(a)mailman3.org <mailto:mailman-users@mailman3.org>>)
> escribió:
> >
> >> I think you're confused.
> >>
> >> You already have mail.labbrands.com <http://mail.labbrands.com>
> set up as the MX record for
> >> labbrands.com <http://labbrands.com>. And it has an A record
> with the same address as your
> >> webserver.
> >>
> > I'm working with the hostgator mailserver, we are not running a
> local
> > mailserver.
> >
> > As suggested I added an A record like this:
> > [image: image.png]
> >
> > Is it correct? is it redirecting to my Ip so I can redirect it to my
> > postfix/mm3 server?
> >
> >
> >> So if you're getting e-mail on that domain, there's another e-mail
> >> server running on that IP address. You can't have 2 servers
> on port
> >> 25. In that case, as has been noted before, you'll need to setup a
> >> relay in that server, not a firewall redirect. Depending on
> your MTA,
> >> you would need to relay to your internal server. And make sure
> your
> >> firewall setup allows your MTA to do this.
> >>
> >> We can see it's EXIM:
> >>
> >> telnet mail.labbrands.com <http://mail.labbrands.com> 25
> >> Trying 192.185.51.89...
> >> Connected to mail.labbrands.com <http://mail.labbrands.com>
> (192.185.51.89).
> >> Escape character is '^]'.
> >> help
> >> 220-gator4137.hostgator.com
> <http://220-gator4137.hostgator.com> ESMTP Exim 4.87 #1 Thu, 03
> Aug 2017 15:47:48
> >> -0500
> >> 220-We do not authorize the use of this system to transport
> unsolicited,
> >> 220 and/or bulk e-mail.
> >> 214-Commands supported:
> >> 214 AUTH STARTTLS HELO EHLO MAIL RCPT DATA NOOP QUIT RSET HELP
> >> quit
> >> 221 gator4137.hostgator.com <http://gator4137.hostgator.com>
> closing connection
> >> Connection closed by foreign host.
> >>
> >> Or, consolidate all your e-mail to one server - which is a lot
> easier to
> >> manage unless you have a really big operation. Postfix is
> probably the
> >> right choice, but requires more setup.
> >>
> >> Although Mailman3 configuration is not well documented (as you've
> >> discovered), you may want to get help from someone with more
> general
> >> network and mail experience. You're now into territory that is, as
> >> Simon indicated, not Mailman-specific.
> >>
> >> On 03-Aug-17 16:28, Rafael Mora wrote:
> >>> El jue., 3 ago. 2017 a las 15:26, Mark Sapiro
> (<mark(a)msapiro.net <mailto:mark@msapiro.net>>)
> >> escribió:
> >>>> On 08/03/2017 01:22 PM, Rafael Mora wrote:
> >>>>> El jue., 3 ago. 2017 a las 15:18, Mark Sapiro
> (<mark(a)msapiro.net <mailto:mark@msapiro.net>>)
> >>>> escribió:
> >>>>>> You need to forward port 25 for SMTP mail delivery and if
> you want the
> >>>>>> web UI (Postorius and HyperKitty) accessible from the
> outside, port 80
> >>>>>> for http and/or port 443 for https
> >>>>>> <
> >>>>>>
> >>
> https://www.iana.org/assignments/service-names-port-numbers/service-names-p…
> >>>>>>> .
> >>>>> I mean for incoming mail redirected from my hostgator
> hosting, because
> >>>> when
> >>>>> I suscribe an email address MM3 sends a confirmation email,
> so I have
> >> to
> >>>>> reply to be suscribed to the list.
> >>>> As I said, for mail delivery you need to forward port 25 to
> the Mailman
> >>>> server.
> >>>>
> >>> Ok so I'll redirect in my zentyal firewall the port 25 to my local
> >> centos7
> >>> with postfix and MM3 with IP 192.168.1.42. Thanks Mark.
> >>>
> >>>
> >>>> --
> >>>> Mark Sapiro <mark(a)msapiro.net <mailto:mark@msapiro.net>>
> The highway is for gamblers,
> >>>> San Francisco Bay Area, California better use your sense -
> B. Dylan
> >>>> _______________________________________________
> >>>> Mailman-users mailing list
> >>>> mailman-users(a)mailman3.org <mailto:mailman-users@mailman3.org>
> >>>>
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> >>>>
> >> --
> >> This communication may not represent my employer's views,
> >> if any, on the matters discussed.
> >>
> >> _______________________________________________
> >> Mailman-users mailing list
> >> mailman-users(a)mailman3.org <mailto:mailman-users@mailman3.org>
> >>
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> >>
>
> --
> This communication may not represent my employer's views,
> if any, on the matters discussed.
>
> _______________________________________________
> Mailman-users mailing list
> mailman-users(a)mailman3.org <mailto:mailman-users@mailman3.org>
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>
> --
>
> Atentamente / Best Regards
>
> Ing. Rafael Mora
>
--
This communication may not represent my employer's views,
if any, on the matters discussed.
6 years, 9 months
Re: Member Issue Discovered
by Brian Carpenter
On 10/19/20 11:32 PM, Mark Sapiro wrote:
> On 10/19/20 7:42 PM, Brian Carpenter wrote:
>> On 10/19/20 10:23 PM, Mark Sapiro wrote:
>>> And the import21 created the user and address records for this user.
>> Does the same thing for a new subscriber as well. So there is no pathway
>> to change a real name that is associated with an email address. None.
>> Zilch. Mailman 2 made this so easy and Mailman 3 made it impossible. I
>> will let someone else file the bug report. You don't really think that
>> this is an issue which means it will be years before it is addressed. So
>> I will save me some time. I will learn to live with it.
>
> I didn't say I didn't think it was a real issue. I mostly questioned
> whether a new subscription should change an existing display_name. I
> agree that there should be a way for a user to change the display_name
> associated with her address. I'm not so sure about a list admin.
Respectively, I think you are asking the wrong question here. The real
question is why isn't a display_name being removed when a list
subscriber is unsubscribed. Also list members being forced into a
powerless role by Mailman 3's architecture have no way of changing a
name except through a list owner unless they register an account but
then again, *even that doesn't allow them to change their display name*.
There are also other use cases in which a list owner will want to
identify portions of their membership roster through the use of the
display_name.
Most list members interact with a Mailing list program via email. I know
you understand that. However it is List owners that are the "real" power
users of Mailman and so the admin interface should be designed in such a
way that empowers List owners. Taking the ability away from List owners
(and List members to make), what should be a simple name change, weakens
them.
> And in any case, I'm only one person. I'm not the only one deciding
> what's important and what's not. I only decide what I want to work on,
> not what anyone else thinks is important or wants to work on, so even if
> I think something is not worth doing, that doesn't mean it won't get
> done, and once again for emphasis, I do thing a user should be able to
> change the display name associated with her address(es).
Again I think we are missing something here from this conversation: data
is not being removed. A list subscriber being removed (unsubscribed)
should have their information totally removed. It is not. I now have one
list owner who realizes this and it is not good. How does this not
violate GDPR? This is why I think all the mailman developers ought to
make this issue a high priority. It is creepy to see a name returned out
of nowhere when someone resubscribes to a list without associating a
name with a second resubscribing. I think this puts List owners at a
disadvantage.
>
> Mailman 3 is totally different from Mailman 2.1 in this respect. Mailman
> 2.1 had no concept of user. All it knew was addresses subscribed to
> lists and an address subscribed to one list had no connection to the
> same address subscribed to another list or being an owner or moderator
> of a list.
I think this should still be the case for non-registered list members.
List owners/moderators are different. They HAVE to be registered users
of Postorius/Affinity in order to manage/moderate a list. I wonder what
the majority of list members' scenarios are. Is the majority scenario in
play today the one where most list members are only subscribed to a
single list on a single mailman instance? Or is it the scenario that is
in majority use the one where a single list member is subscribed to
multiple lists? I think that it is important to find out because it
should govern the development process to a certain degree. This single
approach of having registered user account with multiple associated
email accounts elevates one scenario at the expense of the other. It
also assumes/requires that list members have a registered user account
and I can tell you from my experience that is not going to happen. The
majority of mailman 2 users will not registered with a Mailman 3 admin
interface. Some will of course, the majority will not. Maybe the
behavior should and will change but it will take years for that to
happen. So what I am challenging and questioning here is the approach
where such assumptions are being made already that results in the
weakening of list owners.
>
> Mailman 3 does have a concept of user and addresses belonging to a user.
> This complicates things in some ways. In Mailman 2.1 we could have "The
> Boss <user(a)example.com" as a member of one list and "Just a Peon
> <user(a)example.com>" as a member of another list. In Mailman 3 that is
> not possible unless the addresses are tweaked in some way to make them
> different.
Perhaps there is a better approach here to handle single member
w/multiple subscriptions that doesn't hurt the ability of List owners to
serve the needs of list members who do not have a registered user
account and prevents the removal of data when a SINGLE user unsubscribes
from a list.
>
> You don't seem to be concerned about the case where I subscribe to a
> second list with a different display name and am surprised to find my
> display name changed on the first list, but it's something that I have
> to consider.
I am not concerned because I think such cases are in the extreme
minority. Are you saying that people are now going to be using different
real names associated with their other subscriptions as well and that is
going to be so needed it warrants this complex system that you have come
up with? If someone whats to have a different name associated with a
different subscription/email address then I am going to tell them create
a new user account and use a good password management program to keep
track of their logins. Right now I think this minority use case scenario
is negatively impacting other administrative tasks of Mailman,
particularly in the removal of user data when someone unsubscribes from
a list. Its the use of a square peg being forced into a round hole.
Except in this case most of the holes are round but by golly, we are
going to use that square peg regardless!
>
> As far as filing or not filing an issue, issues in the GitLab trackers
> are how we track these things. Threads on mailing lists are appropriate
> for discussion of issues, but if something is going to get changed or
> fixed, an issue in the tracker is the way to ensure it doesn't get put
> aside and forgotten. If this is as important to you as it seems to be,
> please file the issue.
I will but what is disturbing is that you don't think that the
non-removal of user data in cases of unsubscribing is not important or
even an issue.
--
Brian Carpenter
Harmonylists.com
Emwd.com
3 years, 6 months
Template path
by Seth Seeger
Hello everyone,
I’m running Mailman from Docker containers, without much customization. (Postorius 1.3.3, Core 3.3.1, API 3.1, Python 3.8.5). I’ve created a list:user:notice:welcome template, and when I try to confirm a subscription, I get an error.
I followed previous emails on this list which told me to set POSTORIUS_TEMPLATE_BASE_URL in /opt/mailman/web/settings_local.py, but that doesn’t seem to make any difference. (I set it to a different value and the error message didn’t change. I also tried setting it in /opt/mailman/core/settings_local.py to no effect. After each change, I restarted mailman-web and mailman-core containers.)
From within mailman-web and mailman-core containers, I can reach “mailman-web:8000”.
In the webpage, I see: Mailman REST API not available. Please start Mailman core.
In /opt/mailman/core/var/logs/mailman.log, I get:
[17/Sep/2020:16:23:29 +0000] "GET /3.1/lists/xxx.example.com HTTP/1.1" 200 416 "-" "GNU Mailman REST client v3.3.1"
Sep 17 16:23:35 2020 (48) deque:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse
response.begin()
File "/usr/lib/python3.8/http/client.py", line 307, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.8/socket.py", line 669, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen
retries = retries.increment(
File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 403, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 428, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 335, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/mailman/app/workflow.py", line 69, in __next__
return step()
File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py", line 332, in _step_do_subscription
self.member = self.mlist.subscribe(
File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/lib/python3.8/site-packages/mailman/model/mailinglist.py", line 505, in subscribe
notify(SubscriptionEvent(
File "/usr/lib/python3.8/site-packages/zope/event/__init__.py", line 32, in notify
subscriber(event)
File "/usr/lib/python3.8/site-packages/mailman/app/membership.py", line 176, in handle_SubscriptionEvent
send_welcome_message(mlist, member, member.preferred_language)
File "/usr/lib/python3.8/site-packages/mailman/app/notifications.py", line 51, in send_welcome_message
welcome_message = wrap(getUtility(ITemplateLoader).get(
File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 188, in get
contents = getUtility(ITemplateManager).get(
File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 110, in get
contents = protocols.get(actual_uri, **auth)
File "/usr/lib/python3.8/site-packages/mailman/utilities/protocols.py", line 38, in get
response = requests.get(url, timeout=REQUEST_TIMEOUT, **kws)
File "/usr/lib/python3.8/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.8/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 529, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5)
[2020-09-17 16:23:35 +0000] [48] [ERROR] Socket error processing request.
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse
response.begin()
File "/usr/lib/python3.8/http/client.py", line 307, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.8/socket.py", line 669, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen
retries = retries.increment(
File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 403, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 428, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 335, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 135, in handle
self.handle_request(listener, req, client, addr)
File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 191, in handle_request
six.reraise(*sys.exc_info())
File "/usr/lib/python3.8/site-packages/gunicorn/six.py", line 625, in reraise
raise value
File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 50, in wrapper
rtn = function(*args, **kws)
File "/usr/lib/python3.8/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__
return super().__call__(environ, start_response)
File "/usr/lib/python3.8/site-packages/falcon/api.py", line 269, in __call__
responder(req, resp, **params)
File "/usr/lib/python3.8/site-packages/mailman/rest/sub_moderation.py", line 88, in on_post
self._registrar.confirm(self._token)
File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py", line 559, in confirm
list(workflow)
File "/usr/lib/python3.8/site-packages/mailman/app/workflow.py", line 69, in __next__
return step()
File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py", line 332, in _step_do_subscription
self.member = self.mlist.subscribe(
File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/lib/python3.8/site-packages/mailman/model/mailinglist.py", line 505, in subscribe
notify(SubscriptionEvent(
File "/usr/lib/python3.8/site-packages/zope/event/__init__.py", line 32, in notify
subscriber(event)
File "/usr/lib/python3.8/site-packages/mailman/app/membership.py", line 176, in handle_SubscriptionEvent
send_welcome_message(mlist, member, member.preferred_language)
File "/usr/lib/python3.8/site-packages/mailman/app/notifications.py", line 51, in send_welcome_message
welcome_message = wrap(getUtility(ITemplateLoader).get(
File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 188, in get
contents = getUtility(ITemplateManager).get(
File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 110, in get
contents = protocols.get(actual_uri, **auth)
File "/usr/lib/python3.8/site-packages/mailman/utilities/protocols.py", line 38, in get
response = requests.get(url, timeout=REQUEST_TIMEOUT, **kws)
File "/usr/lib/python3.8/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.8/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 529, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5)
What’s the next step to get templates working?
Thank you,
Seth
3 years, 8 months
Re: Member Issue Discovered
by Brian Carpenter
On 11/4/20 12:04 PM, Stephen J. Turnbull wrote:
> Brian Carpenter writes:
>
> > If you mean if they don't have a Django user account, they can't
> > unsubscribe?
>
> The Django user account is more or less a proxy for the underlying
> Mailman User object, which is what we're concerned with here because
> that's where the data you want deleted is stored.
>
> > I think that is true if they are wanting to unsubscribe via
> > the web interface. But sending an email to
> > listname-unsubscribe@listdomain allows them to unsubscribe without such
> > user authentication. Isn't that correct?
>
> They don't need a Django password or social auth, but they'll still
> have to do the OTK dance. I don't see a reason to distinguish the
> methods of authentication. They all reduce to the OTK dance to prove
> ownership of a mailbox, then optional *delegation* to a password in
> Django or social auth via Gmail etc.
Please speak in simple terms. I am pretty sure my questions were
straightforward but I have a sneaky suspicion you convoluted the issue
with these replies. I feel like my questions were not answered.
>
> I understand that your users don't understand this or care to. Thing
> is, some of *our* users care about the features that this architecture
> enables.
I think everyone should care that data should be removed when there is
an expectation that it is being removed.
>
> > My intent in our changes is to give the list owner and member
> > exactly what they expect: when they explicitly remove a
> > subscription and/or user profile, that they expect their data to be
> > totally removed.
>
> That's fine, and you're welcome to implement that in Affinity. In
> fact, as I understand it, you have already done so. *I'm* not saying
> *nobody* should implement that.
>
> I'm saying that *Mailman* shouldn't, because a lot of subscribers to
> Mailman lists won't like it. The whole point of Mailman 3 is to cater
> to users who want powerful control over their subscriptions. For
> Mailman (Postorius), prompting the user "If you delete this
> subscription, you will have no subscriptions linked to this account.
> Would you like to delete the account as well?" (as I have proposed
> three times now) is the right way to go. This can be implemented via
> email with a second OTK.
Since the account is a django account, I don't see why unsubscribing
from a list has anything to do with that account except removing the
user's prefer settings for a list that they are no longer subscribed to.
For me I made this issue about those list members who unsubscribe from a
single list AND does not have a django user account. This discussion has
expanded to all kinds of scenarios.
Those who are the power users are not in play here. I am sure they
intend to to keep their subscriptions active and their django account
open. I am talking about those who want to leave a list or wants to
remove their django user account permanently.
>
> > I can easily picture a scenario where an older list has a number of
> > list members move to on to other things in life
>
> And I can picture a scenario where they take a summer break and
> reactivate a decade later. I would be *pissed off* if my data got
> deleted in that scenario. (Yes, I've done that in the five year
> variant, if not the full decade.)
Why would you be mad? Am I missing something here? The data that is
being kept are a few fields of data (email address, username, password,
real-name, and some list preferences). Why would anyone expect to have
such inactive accounts kept perpetually? Regardless, I am not talking
about removing accounts/subscriptions without a user's permission. I am
talking about when a user either does it themselves or asks a List Owner
to do so.
>
> > and even abandon their email accounts totally.
>
> Which is a totally different scenario. I deny your "imperative" in
> the former case; inactivity from the point of view of a list owner is
> (usually) not abandonment. I question it in the latter, because
> "abandon" is an inference drawn from lack of activity or bounces,
> either of which could be inadvertant (respectively the "summer break"
> scenario, and separation from an employer).
I worded that poorly. But remember if an email address that no longer
exists (because a list member never unsubscribed from a list and closed
their email provider account anyways) is removed due to bounces (as it
should), that user data still exists. I think that is not good and such
a scenario happens a lot.
>
> > In those cases it is imperative that those list addresses are
> > removed either via bounce processing or list owner intervention and
> > that no legacy data on such members remain.
>
> But how do you propose to identify "legacy data"? In Mailman 3,
> members are *people*, not addresses. Do you really want it to be the
> case that if Albert signs up with albert(a)example.com, and later gets
> fired by example.com, then all his subscriptions get bounce-cancelled,
> and his User profile gets irreversibly trashed? Or some large email
> provider hosting lots of posters decides to suddenly implement some
> spam protection that causes a ton of bounces (as DMARC p=reject did in
> 2014) and hundreds of users have their accounts trashed?
It worked for Mailman 2 right? What happened in those cases? They just
got resubscribed again. As you said, unsubscribing doesn't impact a
django account anyways. So I don't understand your point here. Again I
am talking about the scenario where a user is requesting to be removed
or does the removal themselves and expects their data to be removed.
> You just can't know without asking Albert what he wants done in his
> case and you can be quite sure you're doing the wrong thing in the
> DMARC-like case. In either case, resubscribing is a click per list if
> his data is retained (and Albert needs an OTK confirmation of another
> address). It's an annoying session of duplicating his configuration
> (including passwords and social auth links) if not, and probably some
> months of discovering that a configuration that built up over years
> wasn't accurately reproduced for some lists.
Apples and oranges here. For public lists and a user who does not have a
Affinity/Django account, it is just a click per list. For those who have
a django/Affinity account, well there account should still be there
since they never asked for it to be removed or did not remove it themselves.
Personally I think I am going to implement some sort of backup
generation tool for an Affinity account holder where they can download
their profile/list settings in a json or equivalent format before
account deletion. Then if they wish to create their account, they can
rebuild their profile from that backup.
>
> List owners, of course, can do what they want with their lists. If
> they want to add automation for their subscribers that simplify the
> lives of people who have simple needs, that's not something Mailman
> can, should, or will try to stop. That's *why* I support efforts like
> Affinity, Empathy, and Harmony.
>
> Steve
I personally intend to get into the "power-user" market. That is the #1
reason for creating my UIs. I have that flexibility.
--
Brian Carpenter
Harmonylists.com
Emwd.com
3 years, 6 months
Re: Post installation issues (recipients refused) with Exim4 in a virtual host environment
by Kyriakos Terzopoulos
Please also find below the mailman conf command:
(venv) mailman@hestia:~/venv/bin$ ./mailman conf
[ARC] authserv_id:
[ARC] dkim: yes
[ARC] dmarc: yes
[ARC] domain:
[ARC] enabled: no
[ARC] privkey:
[ARC] selector:
[ARC] sig_headers: From, Sender, Reply-To, Subject, Date, Message-ID, To,
Cc, MIME-Version, Content-Type, Content-Transfer-Encoding, Content-ID,
Content-Description, Resent-Date, Resent-From, Resent-Sender, Resent-To,
Resent-Cc, Resent-Message-ID, In-Reply-To, References, List-Id, List-Help,
List-Unsubscribe, List-Subscribe, List-Post, List-Owner, List-Archive
[ARC] trusted_authserv_ids:
[antispam] header_checks:
[antispam] jump_chain: hold
[archiver.hyperkitty] class: mailman_hyperkitty.Archiver
[archiver.hyperkitty] clobber_date: maybe
[archiver.hyperkitty] clobber_skew: 1d
[archiver.hyperkitty] configuration: /etc/mailman3/mailman-hyperkitty.cfg
[archiver.hyperkitty] enable: yes
[archiver.mail_archive] class: mailman.archiving.mailarchive.MailArchive
[archiver.mail_archive] clobber_date: maybe
[archiver.mail_archive] clobber_skew: 1d
[archiver.mail_archive] configuration: python:mailman.config.mail_archive
[archiver.mail_archive] enable: no
[archiver.mhonarc] class: mailman.archiving.mhonarc.MHonArc
[archiver.mhonarc] clobber_date: maybe
[archiver.mhonarc] clobber_skew: 1d
[archiver.mhonarc] configuration: python:mailman.config.mhonarc
[archiver.mhonarc] enable: no
[archiver.prototype] class: mailman.archiving.prototype.Prototype
[archiver.prototype] clobber_date: maybe
[archiver.prototype] clobber_skew: 1d
[archiver.prototype] configuration: changeme
[archiver.prototype] enable: yes
[bounces] register_bounces_every: 15m
[database] class: mailman.database.postgresql.PostgreSQLDatabase
[database] debug: no
[database] url: postgresql://mailman:XXXXXXXXXX@localhost/mailman
[devmode] enabled: no
[devmode] recipient:
[devmode] testing: no
[devmode] wait: 60s
[digests] mime_digest_keep_headers:
Date From To Cc Subject Message-ID Keywords
In-Reply-To References Content-Type MIME-Version
Content-Transfer-Encoding Precedence Reply-To
Message List-Post
[digests] plain_digest_keep_headers:
Message Date From
Subject To Cc
Message-ID Keywords
Content-Type
[dmarc] cache_lifetime: 7d
[dmarc] org_domain_data_url:
https://publicsuffix.org/list/public_suffix_list.dat
[dmarc] resolver_lifetime: 5s
[dmarc] resolver_timeout: 3s
[language.ar] charset: utf-8
[language.ar] description: Arabic
[language.ar] enabled: yes
[language.ast] charset: utf-8
[language.ast] description: Asturian
[language.ast] enabled: yes
[language.bg] charset: utf-8
[language.bg] description: Bulgarian
[language.bg] enabled: yes
[language.ca] charset: utf-8
[language.ca] description: Catalan
[language.ca] enabled: yes
[language.cs] charset: utf-8
[language.cs] description: Czech
[language.cs] enabled: yes
[language.da] charset: utf-8
[language.da] description: Danish
[language.da] enabled: yes
[language.de] charset: utf-8
[language.de] description: German
[language.de] enabled: yes
[language.el] charset: utf-8
[language.el] description: Greek
[language.el] enabled: yes
[language.en] charset: us-ascii
[language.en] description: English (USA)
[language.en] enabled: yes
[language.es] charset: utf-8
[language.es] description: Spanish
[language.es] enabled: yes
[language.et] charset: utf-8
[language.et] description: Estonian
[language.et] enabled: yes
[language.eu] charset: utf-8
[language.eu] description: Euskara
[language.eu] enabled: yes
[language.fi] charset: utf-8
[language.fi] description: Finnish
[language.fi] enabled: yes
[language.fr] charset: utf-8
[language.fr] description: French
[language.fr] enabled: yes
[language.gl] charset: utf-8
[language.gl] description: Galician
[language.gl] enabled: yes
[language.he] charset: utf-8
[language.he] description: Hebrew
[language.he] enabled: yes
[language.hr] charset: utf-8
[language.hr] description: Croatian
[language.hr] enabled: yes
[language.hu] charset: utf-8
[language.hu] description: Hungarian
[language.hu] enabled: yes
[language.ia] charset: utf-8
[language.ia] description: Interlingua
[language.ia] enabled: yes
[language.it] charset: utf-8
[language.it] description: Italian
[language.it] enabled: yes
[language.ja] charset: utf-8
[language.ja] description: Japanese
[language.ja] enabled: yes
[language.ko] charset: utf-8
[language.ko] description: Korean
[language.ko] enabled: yes
[language.lt] charset: utf-8
[language.lt] description: Lithuanian
[language.lt] enabled: yes
[language.nl] charset: utf-8
[language.nl] description: Dutch
[language.nl] enabled: yes
[language.no] charset: utf-8
[language.no] description: Norwegian
[language.no] enabled: yes
[language.pl] charset: utf-8
[language.pl] description: Polish
[language.pl] enabled: yes
[language.pt] charset: utf-8
[language.pt] description: Portuguese
[language.pt] enabled: yes
[language.pt_BR] charset: utf-8
[language.pt_BR] description: Portuguese (Brazil)
[language.pt_BR] enabled: yes
[language.ro] charset: utf-8
[language.ro] description: Romanian
[language.ro] enabled: yes
[language.ru] charset: utf-8
[language.ru] description: Russian
[language.ru] enabled: yes
[language.sk] charset: utf-8
[language.sk] description: Slovak
[language.sk] enabled: yes
[language.sl] charset: utf-8
[language.sl] description: Slovenian
[language.sl] enabled: yes
[language.sr] charset: utf-8
[language.sr] description: Serbian
[language.sr] enabled: yes
[language.sv] charset: utf-8
[language.sv] description: Swedish
[language.sv] enabled: yes
[language.tr] charset: utf-8
[language.tr] description: Turkish
[language.tr] enabled: yes
[language.uk] charset: utf-8
[language.uk] description: Ukrainian
[language.uk] enabled: yes
[language.vi] charset: utf-8
[language.vi] description: Vietnamese
[language.vi] enabled: yes
[language.zh_CN] charset: utf-8
[language.zh_CN] description: Chinese
[language.zh_CN] enabled: yes
[language.zh_TW] charset: utf-8
[language.zh_TW] description: Chinese (Taiwan)
[language.zh_TW] enabled: yes
[logging.archiver] datefmt: %b %d %H:%M:%S %Y
[logging.archiver] format: %(asctime)s (%(process)d) %(message)s
[logging.archiver] level: info
[logging.archiver] path: mailman.log
[logging.archiver] propagate: no
[logging.bounce] datefmt: %b %d %H:%M:%S %Y
[logging.bounce] format: %(asctime)s (%(process)d) %(message)s
[logging.bounce] level: info
[logging.bounce] path: bounce.log
[logging.bounce] propagate: no
[logging.config] datefmt: %b %d %H:%M:%S %Y
[logging.config] format: %(asctime)s (%(process)d) %(message)s
[logging.config] level: info
[logging.config] path: mailman.log
[logging.config] propagate: no
[logging.database] datefmt: %b %d %H:%M:%S %Y
[logging.database] format: %(asctime)s (%(process)d) %(message)s
[logging.database] level: warn
[logging.database] path: mailman.log
[logging.database] propagate: no
[logging.debug] datefmt: %b %d %H:%M:%S %Y
[logging.debug] format: %(asctime)s (%(process)d) %(message)s
[logging.debug] level: info
[logging.debug] path: debug.log
[logging.debug] propagate: no
[logging.error] datefmt: %b %d %H:%M:%S %Y
[logging.error] format: %(asctime)s (%(process)d) %(message)s
[logging.error] level: info
[logging.error] path: mailman.log
[logging.error] propagate: no
[logging.fromusenet] datefmt: %b %d %H:%M:%S %Y
[logging.fromusenet] format: %(asctime)s (%(process)d) %(message)s
[logging.fromusenet] level: info
[logging.fromusenet] path: mailman.log
[logging.fromusenet] propagate: no
[logging.http] datefmt: %b %d %H:%M:%S %Y
[logging.http] format: %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
[logging.http] level: info
[logging.http] path: mailman.log
[logging.http] propagate: no
[logging.locks] datefmt: %b %d %H:%M:%S %Y
[logging.locks] format: %(asctime)s (%(process)d) %(message)s
[logging.locks] level: info
[logging.locks] path: mailman.log
[logging.locks] propagate: no
[logging.mischief] datefmt: %b %d %H:%M:%S %Y
[logging.mischief] format: %(asctime)s (%(process)d) %(message)s
[logging.mischief] level: info
[logging.mischief] path: mailman.log
[logging.mischief] propagate: no
[logging.plugins] datefmt: %b %d %H:%M:%S %Y
[logging.plugins] format: %(asctime)s (%(process)d) %(message)s
[logging.plugins] level: info
[logging.plugins] path: plugins.log
[logging.plugins] propagate: no
[logging.root] datefmt: %b %d %H:%M:%S %Y
[logging.root] format: %(asctime)s (%(process)d) %(message)s
[logging.root] level: info
[logging.root] path: mailman.log
[logging.root] propagate: no
[logging.runner] datefmt: %b %d %H:%M:%S %Y
[logging.runner] format: %(asctime)s (%(process)d) %(message)s
[logging.runner] level: info
[logging.runner] path: mailman.log
[logging.runner] propagate: no
[logging.smtp] datefmt: %b %d %H:%M:%S %Y
[logging.smtp] every: $msgid smtp to $listname for $recip recips, completed
in $time seconds
[logging.smtp] failure: $msgid delivery to $recip failed with code
$smtpcode, $smtpmsg
[logging.smtp] format: %(asctime)s (%(process)d) %(message)s
[logging.smtp] level: info
[logging.smtp] path: smtp.log
[logging.smtp] propagate: no
[logging.smtp] refused: $msgid post to $listname from $sender, $size bytes,
$refused failures
[logging.smtp] success: $msgid post to $listname from $sender, $size bytes
[logging.subscribe] datefmt: %b %d %H:%M:%S %Y
[logging.subscribe] format: %(asctime)s (%(process)d) %(message)s
[logging.subscribe] level: info
[logging.subscribe] path: mailman.log
[logging.subscribe] propagate: no
[logging.task] datefmt: %b %d %H:%M:%S %Y
[logging.task] format: %(asctime)s (%(process)d) %(message)s
[logging.task] level: info
[logging.task] path: mailman.log
[logging.task] propagate: no
[logging.vette] datefmt: %b %d %H:%M:%S %Y
[logging.vette] format: %(asctime)s (%(process)d) %(message)s
[logging.vette] level: info
[logging.vette] path: mailman.log
[logging.vette] propagate: no
[mailman] anonymous_list_keep_headers: ^x-mailman- ^x-content-filtered-by:
^x-topics:
^x-ack: ^x-beenthere: ^x-list-administrivia: ^x-spam-
[mailman] cache_life: 7d
[mailman] check_max_size_on_filtered_message: no
[mailman] default_language: en
[mailman] email_commands_max_lines: 10
[mailman] filter_report: no
[mailman] filtered_messages_are_preservable: no
[mailman] hold_digest: no
[mailman] html_to_plain_text_command: /usr/bin/lynx -dump $filename
[mailman] layout: local
[mailman] listname_chars: [-_.0-9a-z]
[mailman] masthead_threshold: 4
[mailman] moderator_request_life: 180d
[mailman] noreply_address: noreply
[mailman] pending_request_life: 3d
[mailman] post_hook:
[mailman] pre_hook:
[mailman] run_tasks_every: 1h
[mailman] sender_headers: from from_ reply-to sender
[mailman] site_owner: kterzopoulos(a)XXXXXX.com
[mta] configuration: python:mailman.config.exim4
[mta] delivery_retry_period: 5d
[mta] incoming: mailman.mta.exim4.LMTP
[mta] lmtp_host: localhost
[mta] lmtp_port: 8024
[mta] max_autoresponses_per_day: 10
[mta] max_delivery_threads: 0
[mta] max_recipients: 500
[mta] max_sessions_per_connection: 0
[mta] outgoing: mailman.mta.deliver.deliver
[mta] remove_dkim_headers: no
[mta] smtp_host: mail.myproject.eu
[mta] smtp_pass:
[mta] smtp_port: 25
[mta] smtp_secure_mode: smtp
[mta] smtp_user:
[mta] smtp_verify_cert: yes
[mta] smtp_verify_hostname: yes
[mta] verp_confirm_format: $address+$cookie
[mta] verp_confirm_regexp: ^(.*<)?(?P<addr>[^+]+?)\+(?P<cookie>[^@]+)@.*$
[mta] verp_confirmations: yes
[mta] verp_delimiter: +
[mta] verp_delivery_interval: 1
[mta] verp_format: ${bounces}+${local}=${domain}
[mta] verp_personalized_deliveries: yes
[mta] verp_probe_format: $bounces+$token@$domain
[mta] verp_probe_regexp: ^(?P<bounces>[^+]+?)\+(?P<token>[^@]+)@.*$
[mta] verp_probes: no
[mta] verp_regexp:
^(?P<bounces>[^+]+?)\+(?P<local>[^=]+)=(?P<domain>[^@]+)@.*$
[nntp] gatenews_every: 5m
[nntp] host:
[nntp] password:
[nntp] port:
[nntp] remove_headers:
nntp-posting-host nntp-posting-date x-trace
x-complaints-to xref date-received posted
posting-version relay-version received
[nntp] rewrite_duplicate_headers:
To X-Original-To
CC X-Original-CC
Content-Transfer-Encoding X-Original-Content-Transfer-Encoding
MIME-Version X-MIME-Version
[nntp] user:
[passwords] configuration: python:mailman.config.passlib
[passwords] password_length: 8
[paths.dev] archive_dir: $var_dir/archives
[paths.dev] bin_dir: $argv
[paths.dev] cache_dir: $var_dir/cache
[paths.dev] data_dir: $var_dir/data
[paths.dev] etc_dir: $var_dir/etc
[paths.dev] ext_dir: $var_dir/ext
[paths.dev] list_data_dir: $var_dir/lists
[paths.dev] lock_dir: $var_dir/locks
[paths.dev] lock_file: $lock_dir/master.lck
[paths.dev] log_dir: $var_dir/logs
[paths.dev] messages_dir: $var_dir/messages
[paths.dev] pid_file: $var_dir/master.pid
[paths.dev] queue_dir: $var_dir/queue
[paths.dev] template_dir: $var_dir/templates
[paths.dev] var_dir: $cfg_file/../..
[paths.fhs] archive_dir: $var_dir/archives
[paths.fhs] bin_dir: /sbin
[paths.fhs] cache_dir: $var_dir/cache
[paths.fhs] data_dir: $var_dir/data
[paths.fhs] etc_dir: /etc
[paths.fhs] ext_dir: /etc/mailman.d
[paths.fhs] list_data_dir: $var_dir/lists
[paths.fhs] lock_dir: /var/lock/mailman
[paths.fhs] lock_file: $lock_dir/master.lck
[paths.fhs] log_dir: /var/log/mailman
[paths.fhs] messages_dir: $var_dir/messages
[paths.fhs] pid_file: /var/run/mailman/master.pid
[paths.fhs] queue_dir: /var/spool/mailman
[paths.fhs] template_dir: $var_dir/templates
[paths.fhs] var_dir: /var/lib/mailman
[paths.here] archive_dir: $var_dir/archives
[paths.here] bin_dir: $argv
[paths.here] cache_dir: $var_dir/cache
[paths.here] data_dir: $var_dir/data
[paths.here] etc_dir: $var_dir/etc
[paths.here] ext_dir: $var_dir/ext
[paths.here] list_data_dir: $var_dir/lists
[paths.here] lock_dir: $var_dir/locks
[paths.here] lock_file: $lock_dir/master.lck
[paths.here] log_dir: $var_dir/logs
[paths.here] messages_dir: $var_dir/messages
[paths.here] pid_file: $var_dir/master.pid
[paths.here] queue_dir: $var_dir/queue
[paths.here] template_dir: $var_dir/templates
[paths.here] var_dir: $cwd/var
[paths.local] archive_dir: $var_dir/archives
[paths.local] bin_dir: $argv
[paths.local] cache_dir: $var_dir/cache
[paths.local] data_dir: $var_dir/data
[paths.local] etc_dir: $var_dir/etc
[paths.local] ext_dir: $var_dir/ext
[paths.local] list_data_dir: $var_dir/lists
[paths.local] lock_dir: $var_dir/locks
[paths.local] lock_file: $lock_dir/master.lck
[paths.local] log_dir: $var_dir/logs
[paths.local] messages_dir: $var_dir/messages
[paths.local] pid_file: $var_dir/master.pid
[paths.local] queue_dir: $var_dir/queue
[paths.local] template_dir: $var_dir/templates
[paths.local] var_dir: /opt/mailman/venv
[plugin.master] class:
[plugin.master] component_package:
[plugin.master] configuration:
[plugin.master] enabled: no
[runner.archive] class: mailman.runners.archive.ArchiveRunner
[runner.archive] instances: 1
[runner.archive] max_restarts: 10
[runner.archive] path: $QUEUE_DIR/$name
[runner.archive] sleep_time: 1s
[runner.archive] start: yes
[runner.bad] class: mailman.runners.fake.BadRunner
[runner.bad] instances: 1
[runner.bad] max_restarts: 10
[runner.bad] path: $QUEUE_DIR/$name
[runner.bad] sleep_time: 1s
[runner.bad] start: no
[runner.bounces] class: mailman.runners.bounce.BounceRunner
[runner.bounces] instances: 1
[runner.bounces] max_restarts: 10
[runner.bounces] path: $QUEUE_DIR/$name
[runner.bounces] sleep_time: 2m
[runner.bounces] start: yes
[runner.command] class: mailman.runners.command.CommandRunner
[runner.command] instances: 1
[runner.command] max_restarts: 10
[runner.command] path: $QUEUE_DIR/$name
[runner.command] sleep_time: 1s
[runner.command] start: yes
[runner.digest] class: mailman.runners.digest.DigestRunner
[runner.digest] instances: 1
[runner.digest] max_restarts: 10
[runner.digest] path: $QUEUE_DIR/$name
[runner.digest] sleep_time: 1s
[runner.digest] start: yes
[runner.in] class: mailman.runners.incoming.IncomingRunner
[runner.in] instances: 1
[runner.in] max_restarts: 10
[runner.in] path: $QUEUE_DIR/$name
[runner.in] sleep_time: 1s
[runner.in] start: yes
[runner.lmtp] class: mailman.runners.lmtp.LMTPRunner
[runner.lmtp] instances: 1
[runner.lmtp] max_restarts: 10
[runner.lmtp] path:
[runner.lmtp] sleep_time: 1s
[runner.lmtp] start: yes
[runner.nntp] class: mailman.runners.nntp.NNTPRunner
[runner.nntp] instances: 1
[runner.nntp] max_restarts: 10
[runner.nntp] path: $QUEUE_DIR/$name
[runner.nntp] sleep_time: 1s
[runner.nntp] start: yes
[runner.out] class: mailman.runners.outgoing.OutgoingRunner
[runner.out] instances: 1
[runner.out] max_restarts: 10
[runner.out] path: $QUEUE_DIR/$name
[runner.out] sleep_time: 1s
[runner.out] start: yes
[runner.pipeline] class: mailman.runners.pipeline.PipelineRunner
[runner.pipeline] instances: 1
[runner.pipeline] max_restarts: 10
[runner.pipeline] path: $QUEUE_DIR/$name
[runner.pipeline] sleep_time: 1s
[runner.pipeline] start: yes
[runner.rest] class: mailman.runners.rest.RESTRunner
[runner.rest] instances: 1
[runner.rest] max_restarts: 10
[runner.rest] path:
[runner.rest] sleep_time: 1s
[runner.rest] start: yes
[runner.retry] class: mailman.runners.retry.RetryRunner
[runner.retry] instances: 1
[runner.retry] max_restarts: 10
[runner.retry] path: $QUEUE_DIR/$name
[runner.retry] sleep_time: 15m
[runner.retry] start: yes
[runner.shunt] class: mailman.runners.fake.ShuntRunner
[runner.shunt] instances: 1
[runner.shunt] max_restarts: 10
[runner.shunt] path: $QUEUE_DIR/$name
[runner.shunt] sleep_time: 1s
[runner.shunt] start: no
[runner.task] class: mailman.runners.task.TaskRunner
[runner.task] instances: 1
[runner.task] max_restarts: 10
[runner.task] path:
[runner.task] sleep_time: 1m
[runner.task] start: yes
[runner.virgin] class: mailman.runners.virgin.VirginRunner
[runner.virgin] instances: 1
[runner.virgin] max_restarts: 10
[runner.virgin] path: $QUEUE_DIR/$name
[runner.virgin] sleep_time: 1s
[runner.virgin] start: yes
[shell] banner: Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.
[shell] history_file: $var_dir/history.py
[shell] prompt: >>>
[shell] use_ipython: no
[styles] default: legacy-default
[webservice] admin_pass: restpass
[webservice] admin_user: restadmin
[webservice] api_version: 3.1
[webservice] configuration: python:mailman.config.gunicorn
[webservice] hostname: localhost
[webservice] port: 8001
[webservice] show_tracebacks: yes
[webservice] use_https: no
[webservice] workers: 2
*Kyriakos Terzopoulos
*Web developer / e-learning expert
*Tel:*+30 211 213 9858
*Mobile:* +30 694 526 4512
* E-mail: *kyriakos.terzopoulos(a)gmail.com
* Skype:* kyriakos.terzopoulos
Find me on Facebook <http://www.facebook.com/cirrus3d>
Follow me on Twitter <http://twitter.com/#%21/cirrus3d>
On Wed, 11 Oct 2023 at 12:37, Kyriakos Terzopoulos <
kyriakos.terzopoulos(a)gmail.com> wrote:
> Hi,
>
> I have the following in my mailman log
>
> [11/Oct/2023:12:31:39 +0300] "GET /3.1/lists/
> testlist(a)myproject.eu/held/count HTTP/1.1" 200 73 "-" "GNU Mailman REST
> client v3.3.5"
> [11/Oct/2023:12:31:43 +0300] "GET /3.1/lists/testlist.myproject.eu
> HTTP/1.1" 200 425 "-" "GNU Mailman REST client v3.3.5"
> [11/Oct/2023:12:31:43 +0300] "GET /3.1/lists/testlist(a)myproject.eu/config
> HTTP/1.1" 200 3111 "-" "GNU Mailman REST client v3.3.5"
> [11/Oct/2023:12:31:43 +0300] "GET /3.1/lists/
> testlist.myproject.eu/archivers HTTP/1.1" 200 100 "-" "GNU Mailman REST
> client v3.3.5"
> [11/Oct/2023:12:31:51 +0300] "GET /3.1/lists/testlist.myproject.eu
> HTTP/1.1" 200 425 "-" "GNU Mailman REST client v3.3.5"
> Oct 11 12:31:51 2023 (1534165) deque: do_confirm_verify
> Traceback (most recent call last):
> File
> "/opt/mailman/venv/lib/python3.10/site-packages/mailman/app/workflow.py",
> line 69, in __next__
> return step()
> File
> "/opt/mailman/venv/lib/python3.10/site-packages/mailman/app/subscriptions.py",
> line 382, in _step_send_confirmation
> raise StopIteration
> StopIteration
> [11/Oct/2023:12:31:51 +0300] "POST /3.1/members HTTP/1.1" 202 143 "-" "GNU
> Mailman REST client v3.3.5"
> [11/Oct/2023:12:31:51 +0300] "GET /3.1/lists/testlist.myproject.eu
> HTTP/1.1" 200 425 "-" "GNU Mailman REST client v3.3.5"
> [11/Oct/2023:12:31:51 +0300] "GET /3.1/lists/testlist(a)myproject.eu/config
> HTTP/1.1" 200 3111 "-" "GNU Mailman REST client v3.3.5"
> [11/Oct/2023:12:31:51 +0300] "GET /3.1/lists/
> testlist.myproject.eu/archivers HTTP/1.1" 200 100 "-" "GNU Mailman REST
> client v3.3.5"
>
> *Kyriakos Terzopoulos
> *Web developer / e-learning expert
>
> *Tel:*+30 211 213 9858
>
> *Mobile:* +30 694 526 4512
>
> * E-mail: *kyriakos.terzopoulos(a)gmail.com
> * Skype:* kyriakos.terzopoulos
> Find me on Facebook <http://www.facebook.com/cirrus3d>
> Follow me on Twitter <http://twitter.com/#%21/cirrus3d>
>
>
>
> On Wed, 11 Oct 2023 at 11:54, Odhiambo Washington <odhiambo(a)gmail.com>
> wrote:
>
>> On Wed, Oct 11, 2023 at 11:15 AM Kyriakos Terzopoulos <
>> kyriakos.terzopoulos(a)gmail.com> wrote:
>>
>> > Hi,
>> >
>> > I have set up (I believe successfully) and configured the following in a
>> > Virtualhost environment (with HestiaCP):
>> >
>> > - A record for lists.mydomain.com
>> > - Venv
>> > - Mailman core
>> > - Mailman web (run migrate, collectstatic, compress, compilemessages)
>> > Postorius
>> > - gunicorn, hyperkitty
>> > - Nginx
>> > - SMTP exim4 conf
>> >
>> > I can see Posrorius and I have logged in and created my first mailing
>> list:
>> >
>> > [image: Screenshot 2023-10-11 at 10.57.31 AM.png]
>> >
>> > My issue is that even though I have subscribed to my list as you can see
>> > above, I am not receiving the confirmation email. I get a "recipients
>> > refused" error. The smtp log for mailman is as follows:
>> >
>> > Oct 11 10:41:20 2023 (1533666)
>> > <169701007904.1534165.14569329382406660776@localhost> recipients
>> refused:
>> > {'
>> > kyriakos.terzopoulos(a)gmail.com': (550, b'Verification failed for
>> > <testlist-bounces+kyriakos.terzopoulos=gmail.com(a)myproject.eu
>> > >\nUnrouteable
>> > address\nSender verify failed')}
>> >
>> > Oct 11 10:41:20 2023 (1533666)
>> > <169701007904.1534165.14569329382406660776@localhost> smtp to
>> > testlist(a)myproject.eu for 1 recips, completed in 0.26796579360961914
>> > seconds
>> >
>> > Oct 11 10:41:20 2023 (1533666)
>> > <169701007904.1534165.14569329382406660776@localhost> post to
>> > testlist(a)myproject.eu from
>> > testlist-confirm+c9b20c5290bec43147a7e96a6b8cd825daf913d7(a)myproject.eu,
>> > 1667 bytes, 1 failures
>> >
>> > Oct 11 10:41:20 2023 (1533666)
>> > <169701007904.1534165.14569329382406660776@localhost> delivery to
>> > kyriakos.terzopoulos(a)gmail.com failed with code 550, b'Verification
>> failed
>> > for <testlist-bounces+kyriakos.terzopoulos=gmail.com(a)myproject.eu
>> > >\nUnrouteable
>> > address\nSender verify failed'
>> >
>> > I have used the configuration for exim that is here
>> > <
>> >
>> https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h…
>> > >
>> > and
>> > I have also tested with the alternative approach that is documented in
>> >
>>
>> What do you have in the Exim's mainlog?
>>
>>
>> --
>> Best regards,
>> Odhiambo WASHINGTON,
>> Nairobi,KE
>> +254 7 3200 0004/+254 7 2274 3223
>> "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
>> [How to ask smart questions:
>> http://www.catb.org/~esr/faqs/smart-questions.html]
>> _______________________________________________
>> Mailman-users mailing list -- mailman-users(a)mailman3.org
>> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>> Archived at:
>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>>
>> This message sent to kyriakos.terzopoulos(a)gmail.com
>>
>
7 months, 1 week
issue with Hyperkitty
by Ali Alnubani
Hi,
I'm migrating from mailman 2.1.15, I did the following:
* Installed packages
* Created database
* ./manage.py migrate
* mailman create dev(a)example.com<mailto:dev@example.com>
* ./manage.py hyperkitty_import -l dev(a)example.com /usr/local/mailman2/archives/private/dev.mbox/dev.mbox (the archive includes 69 emails).
* ./manage.py update_index_one_list dev(a)example.com
I'm seeing the following trace when I go to http://example.com/hyperkitty/list/dev@example.com/2018/5/:
```
ERROR 2018-08-07 13:54:23,897 19997 django.request Internal Server Error: /hyperkitty/list/dev(a)example.com/2018/5/
Traceback (most recent call last):
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/core/handlers/base.py", line 126, in _get_response
response = self.process_exception_by_middleware(e, request)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/core/handlers/base.py", line 124, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "venv_path/lib/python3.5/site-packages/HyperKitty-1.2.0-py3.5.egg/hyperkitty/lib/view_helpers.py", line 134, in inner
return func(request, *args, **kwargs)
File "venv_path/lib/python3.5/site-packages/HyperKitty-1.2.0-py3.5.egg/hyperkitty/views/mlist.py", line 94, in archives
return _thread_list(request, mlist, threads, extra_context=extra_context)
File "venv_path/lib/python3.5/site-packages/HyperKitty-1.2.0-py3.5.egg/hyperkitty/views/mlist.py", line 123, in _thread_list
return render(request, template_name, context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/shortcuts.py", line 36, in render
content = loader.render_to_string(template_name, context, request, using=using)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/loader.py", line 62, in render_to_string
return template.render(context, request)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/backends/django.py", line 61, in render
return self.template.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 171, in render
return self._render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 937, in render
bit = node.render_annotated(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/loader_tags.py", line 150, in render
return compiled_parent._render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 937, in render
bit = node.render_annotated(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/loader_tags.py", line 62, in render
result = block.nodelist.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 937, in render
bit = node.render_annotated(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/library.py", line 234, in render
return t.render(new_context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 173, in render
return self._render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 937, in render
bit = node.render_annotated(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/defaulttags.py", line 209, in render
nodelist.append(node.render_annotated(context))
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/defaulttags.py", line 309, in render
return nodelist.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 937, in render
bit = node.render_annotated(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/library.py", line 192, in render
output = self.func(*resolved_args, **resolved_kwargs)
File "venv_path/lib/python3.5/site-packages/django_mailman3-1.2.0-py3.5.egg/django_mailman3/templatetags/pagination.py", line 36, in add_to_query_string
return conditional_escape(qs.urlencode())
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/http/request.py", line 524, in urlencode
for v in list_
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/http/request.py", line 524, in <genexpr>
for v in list_
AttributeError: 'int' object has no attribute 'encode'
ERROR 2018-08-07 13:54:23,897 19997 django.request Internal Server Error: /hyperkitty/list/dev(a)example.com/2018/5/
Traceback (most recent call last):
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/core/handlers/base.py", line 126, in _get_response
response = self.process_exception_by_middleware(e, request)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/core/handlers/base.py", line 124, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "venv_path/lib/python3.5/site-packages/HyperKitty-1.2.0-py3.5.egg/hyperkitty/lib/view_helpers.py", line 134, in inner
return func(request, *args, **kwargs)
File "venv_path/lib/python3.5/site-packages/HyperKitty-1.2.0-py3.5.egg/hyperkitty/views/mlist.py", line 94, in archives
return _thread_list(request, mlist, threads, extra_context=extra_context)
File "venv_path/lib/python3.5/site-packages/HyperKitty-1.2.0-py3.5.egg/hyperkitty/views/mlist.py", line 123, in _thread_list
return render(request, template_name, context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/shortcuts.py", line 36, in render
content = loader.render_to_string(template_name, context, request, using=using)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/loader.py", line 62, in render_to_string
return template.render(context, request)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/backends/django.py", line 61, in render
return self.template.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 171, in render
return self._render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 937, in render
bit = node.render_annotated(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/loader_tags.py", line 150, in render
return compiled_parent._render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 937, in render
bit = node.render_annotated(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/loader_tags.py", line 62, in render
result = block.nodelist.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 937, in render
bit = node.render_annotated(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/library.py", line 234, in render
return t.render(new_context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 173, in render
return self._render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 937, in render
bit = node.render_annotated(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/defaulttags.py", line 209, in render
nodelist.append(node.render_annotated(context))
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/defaulttags.py", line 309, in render
return nodelist.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 937, in render
bit = node.render_annotated(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/base.py", line 904, in render_annotated
return self.render(context)
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/template/library.py", line 192, in render
output = self.func(*resolved_args, **resolved_kwargs)
File "venv_path/lib/python3.5/site-packages/django_mailman3-1.2.0-py3.5.egg/django_mailman3/templatetags/pagination.py", line 36, in add_to_query_string
return conditional_escape(qs.urlencode())
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/http/request.py", line 524, in urlencode
for v in list_
File "venv_path/lib/python3.5/site-packages/Django-2.1-py3.5.egg/django/http/request.py", line 524, in <genexpr>
for v in list_
AttributeError: 'int' object has no attribute 'encode'
ERROR 2018-08-07 13:54:25,721 19997 django.server "GET /hyperkitty/list/dev(a)example.com/2018/5/ HTTP/1.1" 500 317983
```
Environment:
```
alembic==1.0.0
arrow==0.12.1
click==6.7
Django==2.1
django-haystack==2.8.1
django-mailman3==1.2.0
django-q==0.9.4
djangorestframework==3.8.2
httplib2==0.11.3
HyperKitty==1.2.0
mailman==3.2.0
mailman-hyperkitty==1.1.1
mailmanclient==3.2.0
Mako==1.0.7
postorius==1.2.0
psycopg2-binary==2.7.5
python-dateutil==2.7.3
requests==2.19.1
requests-oauthlib==1.0.0
SQLAlchemy==1.2.10
urllib3==1.23
uWSGI==2.0.17.1
Whoosh==2.7.4
```
Some of the archives are listed fine tho.
The same trace reproduced with 2 different imported archives.
I'm on Debian stretch and using python 3.5.3.
Can you help please?
Thanks,
Ali
5 years, 9 months