Re: sqlite3 Database "read only" error on mailman_sync job
by Abhilash Raj
On Thu, Sep 5, 2019, at 1:49 PM, Ugnius wrote:
>
> Thank You. This is very valuable information. According documentation I understood that "runserver" should run on some internal port and uwsgi just works like proxy connects to the same as runserver port and serves content to apachi. Because when I was playing with the setup without "runserver" Web interface didn't work.
> Yes, I have stopped "runserver" and page still works fine. :) This didn't solve me database RO issue, but this information was important. Thank You. I am thinking what to do now... Maybe to recreate database (make initiation once again). It could take 2-3 hours. I did not noticed option to export/import lists setting. Or to try find solution how to make sqlite3 multiple
> writable.
I think your problem about locked database might be solved. Like I mentioned, it is possible that those two running server could be stepping over each other toes.
After removing the "runserver" process, I think you shouldn't fall into anymore of those locking issues.
FWIW, it is also highly recommended to not use sqlite for any production use cases. Either postgres or mysql is preferred.
> SQLite version 3.16.2 2017-01-06 16:32:41
> Enter ".help" for usage hints.
>
> BR
> Ugnius
>
> On 5 Sep 2019, at 23:31, Abhilash Raj <maxking(a)asynchronous.in> wrote:
>> On Thu, Sep 5, 2019, at 1:15 PM, Ugnius wrote:
>>>
>>> I do not know... In the setup example settings was something like process number = 2. But I understand at least 1 uwsgi and 1 run server should run?
>>
>> No, only uwsgi needs to run. It is fine if there are two of those, since it probably forks to have 2 worker process. There shouldn't be any "python manage.py runserver" processes running at all, it is meant only for testing purposes.
>>
>> The doc mentions that it is "development" server[1][2].
>>
>> [1]: # Run the Django's "development" server at localhost:8000
>> [2]: http://docs.mailman3.org/en/latest/prodsetup.html#setting-up-django-project
>>
>> Perhaps it should be more descriptive about what does it mean.
>>
>>>
>>> On 5 Sep 2019, at 23:08, Abhilash Raj <maxking(a)asynchronous.in> wrote:
>>>> 6644 ? S 0:12 /usr/bin/uwsgi --master --ini /opt/home/uwsgi.ini
>>>> 6645 ? S 0:00 /usr/bin/python3 /opt/home/manage.py runserver 0:8015
>>>> 6653 ? Sl 636:28 /usr/bin/python3 /opt/home/manage.py runserver 0:8015
>>>> 6656 ? Sl 0:27 /usr/bin/uwsgi --master --ini /opt/home/uwsgi.ini
>>>>
>>>> This fishy, why do you have both uwsgi and "python manage.py runserver" running?
>>>>
>>>> You only need one of those, since they both do the exact same thing. Please remove
>>>> whatever is running the "python manage.py runserver" part and only use uwsgi.
>>>>
>>>> On Thu, Sep 5, 2019, at 1:02 PM, Ugnius S wrote:
>>>>> Thank You, Abhilash, to helping. I have checked the processes, but also I have read, that sqlite3 can run in multi writing mode.
>>>>> May mailman processes are next:
>>>>>
>>>>> 6644 ? S 0:12 /usr/bin/uwsgi --master --ini /opt/home/uwsgi.ini
>>>>> 6645 ? S 0:00 /usr/bin/python3 /opt/home/manage.py runserver 0:8015
>>>>> 6653 ? Sl 636:28 /usr/bin/python3 /opt/home/manage.py runserver 0:8015
>>>>> 6656 ? Sl 0:27 /usr/bin/uwsgi --master --ini /opt/home/uwsgi.ini
>>>>> 6643 ? S 0:00 /usr/bin/python3 /usr/bin/master -C /opt/home/mailman.cfg
>>>>> 659 ? S 0:46 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=in:0:1
>>>>> 6660 ? S 0:42 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=nntp:0:1
>>>>> 6661 ? S 0:42 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=digest:0:1
>>>>> 6662 ? S 0:48 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=pipeline:0:1
>>>>> 6663 ? S 0:44 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=archive:0:1
>>>>> 6664 ? S 0:41 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=command:0:1
>>>>> 6665 ? S 0:01 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=retry:0:1
>>>>> 6666 ? S 0:45 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=virgin:0:1
>>>>> 6667 ? Sl 0:16 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=lmtp:0:1
>>>>> 6668 ? Sl 1:30 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=rest:0:1
>>>>> 6669 ? S 0:42 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=bounces:0:1
>>>>> 6670 ? S 0:49 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=out:0:1
>>>>>
>>>>> Looks like there are no double. Nothing more is running.
>>>>>
>>>>> 2019-09-05, kt, 03:16 Abhilash Raj <maxking(a)asynchronous.in> rašė:
>>>>>> __
>>>>>> On Wed, Sep 4, 2019, at 2:05 PM, Ugnius S wrote:
>>>>>>>
>>>>>>> I have installed using documentation. My linux distribution version has no full package available. I have installed step by step:
>>>>>>> $ sudo pip3 install mailman
>>>>>>> $ sudo pip3 install postorius
>>>>>>> $ sudo pip3 install hyperkitty
>>>>>>> $ sudo pip3 install mailman-hyperkitty
>>>>>>> And of course all mandatory components... Everything without errors, I can drop db files and I can download mailman-suite again and start from empty page again. But it will not guarantee that DB will not become READ ONLY again. Here should be logic. No developing support here?
>>>>>>
>>>>>> Could you be running two separate processes such that the database is locked by one and so other finds it as READ Only? When you find a read only database, can you check if it is open in some other process?
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2019-09-03, an, 19:36 Abhilash Raj <maxking(a)asynchronous.in> rašė:
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Sep 1, 2019, at 3:22 AM,ugniusviln(a)gmail.com wrote:
>>>>>>>> > Hi. All the project installation is under the same user / path and
>>>>>>>> > it works. Archiving working as well, (just one remark: no matter what
>>>>>>>> > to set in postorius, hyperkitty shows list as public)
>>>>>>>> > I have found work around - it is possible to change list as private in
>>>>>>>> > "admin" page.
>>>>>>>> > I started to get issues with runjobs after I have created couple of
>>>>>>>> > new lists with "no archive" option.
>>>>>>>>
>>>>>>>> How did you install all these components. It seems there the installation is broken in weird ways.
>>>>>>>>
>>>>>>>> >
>>>>>>>> > If as owner user from project home I run:
>>>>>>>> >
>>>>>>>> > django-admin mailman_sync
>>>>>>>> >
>>>>>>>> > Traceback (most recent call last):
>>>>>>>> > File
>>>>>>>> > "/home/user/.local/lib/python3.5/site-packages/django/db/backends/utils.py",
>>>>>>>> > line 84, in _execute
>>>>>>>> > return self.cursor.execute(sql, params)
>>>>>>>> > File
>>>>>>>> > "/home/user/.local/lib/python3.5/site-packages/django/db/backends/sqlite3/base.py",
>>>>>>>> > line 383, in execute
>>>>>>>> > return Database.Cursor.execute(self, query, params)
>>>>>>>> > sqlite3.OperationalError: attempt to write a readonly database
>>>>>>>> >
>>>>>>>> > "django-admin runjob sync_mailman " - makes read only errors as well.
>>>>>>>> > "django-admin runjob new_lists_from_mailman"
>>>>>>>> > django.db.utils.OperationalError: attempt to write a readonly database
>>>>>>>> > END TRACEBACK
>>>>>>>> >
>>>>>>>> > I did not find information how to fix this. It was worked without errors
>>>>>>>> > before.
>>>>>>>> > I noticed, that if I stop mailman, "django-admin mailman_sync" starts
>>>>>>>> > to work.
>>>>>>>> > If I start "mailman start", then "django-admin mailman_sync" failing
>>>>>>>> > again.
>>>>>>>> > I did not check other jobs, but I think what is related to database,
>>>>>>>> > will get errors.
>>>>>>>> > Seems mailman locking the database. It is not a file permissions issue.
>>>>>>>> > Mailman db location is: /project_directory/var/data/mailman.db
>>>>>>>> >
>>>>>>>> > if to rename /project_directory/var/data/mailman.db to something,
>>>>>>>> > mailman creates database file size 0, django-admin runjobs start
>>>>>>>> > working, but of course postorius stops working. I did it for testing.
>>>>>>>> > I have returned this database file back, postorius started to work
>>>>>>>> > again, but django-admin runjobs does not work, error: "database read
>>>>>>>> > only".
>>>>>>>> > Please help me how to fix this issue.
>>>>>>>> > _______________________________________________
>>>>>>>> > Mailman-users mailing list -- mailman-users(a)mailman3.org
>>>>>>>> > To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>>>>>>>> > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>>>>>>> >
>>>>>>>>
>>>>>>>> --
>>>>>>>> thanks,
>>>>>>>> Abhilash Raj (maxking)
>>>>>>
>>>>>> --
>>>>>> thanks,
>>>>>> Abhilash Raj (maxking)
>>>>>>
>>>>>>
>>>>
>>>> --
>>>> thanks,
>>>> Abhilash Raj (maxking)
>>>>
>>>>
>>
>> --
>> thanks,
>> Abhilash Raj (maxking)
>>
>>
--
thanks,
Abhilash Raj (maxking)
6 years, 4 months
Re: Mailman 3 localization
by Stephen J. Turnbull
Daniel writes:
> FYI: I did translate some strings a while ago. Have a look
> https://gitlab.com/dadosch/mailman3-german-translation-foo
>
> The reason I didn't made a PR or similar was, that I didn't know where
> :D
I think you attach those messages in
mailman/src/mailman/messages/de/LC_MESSAGES/de.po, and add a README.de
at mailman/src/mailman/messages/de/README.de. The README contains
information about how to participate in the German translation, such
as who is the coordinator (or at least someone willing to read email
about it and bring it to our attention), if you're using a translation
management site, etc. Then submit the PR at
http://gitlab.com/mailman/mailman.
If you are willing to coordinate the German translation, I bet Simon
would be willing to cooperate in merging his translation. It would
certainly be helpful in getting the process going to have an example
of a translation effort for other languages to follow, and it looks
like there's a lot of interest in German. So German may be a good
place to start.
As I mentioned before, the core developers are not particularly
focused on any translation effort. The policy issue about whether to
centralize translation management and/or allow use of sites based on
non-free software is delicate, as mentioned elsewhere, and there is
the issue of coordinating the translation community with release
management. Finally, there is the task of translation for Postorius
and HyperKitty, which may benefit from a use of Django-specific tools
(I'm not familiar with localizing Django applications so that may be
completely stupid :-). So this is going to take time to plan.
Please feel free to lobby us, here and on mailman-developers.
Contribution of effort (planning or coordination) is not necessary but
of course would be welcome. Also, anybody who has participated in
translation efforts who can describe a process that results in
accurate translations while minimizing core developer and translator
pain would be very welcome.
Steve
7 years, 6 months
Re: MM3 Postfix FROM header issue
by Mark Sapiro
On 12/17/23 9:10 AM, eboltz(a)lhtservices.com wrote:
> Mark,
>
> Here is my postconf -n
>
> alias_database = hash:/etc/aliases
> alias_maps = hash:/etc/aliases
> always_add_missing_headers = yes
> append_dot_mydomain = no
> biff = no
> compatibility_level = 2
> default_destination_concurrency_limit = 15
> default_destination_recipient_limit = 30
> header_checks = regexp:/etc/postfix/header_checks
> header_size_limit = 409600
> inet_interfaces = all
> inet_protocols = ipv4
> local_recipient_maps = proxy:unix:passwd.byname $alias_maps hash:/var/lib/mailman3/data/postfix_lmtp
> mailbox_size_limit = 0
> mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
> myhostname = lists.example.com
> mynetworks = 127.0.0.0/8 [::fff:127.0.0.0]/104 [::1]/128
> myorigin = /etc/mailname
> readme_directory = no
> recipient_delimiter = +
> relay_domains = hash:/var/lib/mailman3/data/postfix_domains
> relayhost = [mail.smtp2go.com]:587
> smtp_always_send_ehlo = yes
> smtp_sasl_security_options = noanonymous
> smtp_tls_CApath = /etc/postfix/o365_smtp_cert.pem
> smtp_tls_security_level = may
> smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
> smtp_use_tls = yes
> smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
> smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
> smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
> smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key
> smtpd_tls_security_level = may
> transport_maps = hash:/var/lib/mailman3/data/postfix_lmtp
> unknown_local_recipient_reject_code = 550
> virtual_alias_maps = hash:/var/lib/mailman3/data/postfix_vmap
It looks like you are now following
<https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h…>.
> What would I have to do to ensure that MM3 and postfix (same new server) use the newly created lists.example.org domain, which only exists within Postfix when setting up the Postorius domain and lists?
> I want to have it so that whatever the list name is, for example 'Staff(a)lists.example.org' will route through Postfix correctly.
Have you set an alias_domain for the lists.example.org domain?
What are the contents of /var/lib/mailman3/data/postfix_vmap and
/var/lib/mailman3/data/postfix_lmtp? If you feel it is necessary to
anonymize the domains in these, please only change the tld so we can see
the actual subdomains.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
2 years, 1 month
Re: Docker mailman3 delete social logins, did something change?
by Abhilash Raj
On 9/15/21 7:37 AM, bob B wrote:
> I want to remove the social logins from mailman3 in docker.
Which version tag are you using?
>
> I found this info
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…
>
> but the link to the default shows
> ***************
> INSTALLED_APPS = []
> DEFAULT_APPS = [
> 'hyperkitty',
> 'postorius',
> 'django_mailman3',
> # Uncomment the next line to enable the admin:
> 'django.contrib.admin',
> # Uncomment the next line to enable admin documentation:
> # 'django.contrib.admindocs',
> 'django.contrib.auth',
> 'django.contrib.contenttypes',
> 'django.contrib.sessions',
> 'django.contrib.sites',
> 'django.contrib.messages',
> 'django.contrib.staticfiles',
> 'rest_framework',
> 'django_gravatar',
> 'compressor',
> 'haystack',
> 'django_extensions',
> 'django_q',
> 'allauth',
> 'allauth.account',
> 'allauth.socialaccount',
> ]
> *****************
The link to default is a dynamic one, so it changes as the changes are
made in the repo, unfortunately, this is basically the new stuff that
isn't out yet.
>
>
> However when I go to the bug https://github.com/maxking/docker-mailman/issues/283
>
> it shows
>
> *********
> INSTALLED_APPS = (
> 'hyperkitty',
> 'postorius',
> 'django_mailman3',
> # Uncomment the next line to enable the admin:
> 'django.contrib.admin',
> # Uncomment the next line to enable admin documentation:
> # 'django.contrib.admindocs',
> 'django.contrib.auth',
> 'django.contrib.contenttypes',
> 'django.contrib.sessions',
> 'django.contrib.sites',
> 'django.contrib.messages',
> 'django.contrib.staticfiles',
> 'rest_framework',
> 'django_gravatar',
> 'paintstore',
> 'compressor',
> 'haystack',
> 'django_extensions',
> 'django_q',
> 'allauth',
> 'allauth.account',
> 'allauth.socialaccount',
> 'django_mailman3.lib.auth.fedora',
> 'allauth.socialaccount.providers.openid',
> 'allauth.socialaccount.providers.github',
> 'allauth.socialaccount.providers.gitlab',
> 'allauth.socialaccount.providers.google',
> )
>
> ********
>
>
> The two examples are different so what should it be?
If you want to remove social auth applications, you basically want to
override INSTALLED_APPS configuraiton in your settings_local.py. In the
list above, you want to remove everything after
'django_mailman3.lib.auth.fedora' and set that as INSTALLED_APPS in your
settings_local.py.
>
> I tried to add it in both ways and when I do I get bad gateway and he mailman website is broke. I have not setup any domains yet, just setup the admin password,.
Bad gateway usually will emit some sort of logging error in the mailman
logs at /opt/mailman/web/logs/, what kind of exception are you seeing there?
The configuration above, while enables social auth, shouldn't *break*
the website. There must be errors/exceptions in logs to point in the
direction of what the error might be.
>
>
> For example this breaks my website
>
> cat /opt/mailman/web/settings_local.py
>
> ALLOWED_HOSTS = ['127.0.0.1','localhost','REDACTED','REDACTED','REDACTED','REDACTED','REDACTED']
> #ALLOWED_HOSTS = ['*']
> DEBUG = True
> #DEBUG = False
> EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
> EMAIL_HOST = 'REDACTED'
> EMAIL_PORT = 25
> INSTALLED_APPS = (
> 'hyperkitty',
> 'postorius',
> 'django_mailman3',
> # Uncomment the next line to enable the admin:
> 'django.contrib.admin',
> # Uncomment the next line to enable admin documentation:
> # 'django.contrib.admindocs',
> 'django.contrib.auth',
> 'django.contrib.contenttypes',
> 'django.contrib.sessions',
> 'django.contrib.sites',
> 'django.contrib.messages',
> 'django.contrib.staticfiles',
> 'rest_framework',
> 'django_gravatar',
> 'paintstore',
> 'compressor',
> 'haystack',
> 'django_extensions',
> 'django_q',
> 'allauth',
> 'allauth.account',
> 'allauth.socialaccount',
> 'django_mailman3.lib.auth.fedora',
> 'allauth.socialaccount.providers.openid',
> 'allauth.socialaccount.providers.github',
> 'allauth.socialaccount.providers.gitlab',
> 'allauth.socialaccount.providers.google',
> )
> *************************************************************************
> And I have even removed anything?
>
> What should my /opt/mailman/web/settings_local.py look like?
--
thanks,
Abhilash Raj (maxking)
4 years, 4 months
start_chain not present in migrated lists
by Paul Honig
Hi,
Please let me know where I can post my question if this list is not the
right place.
I'm configuring a mailing list server, migrating from Mailman 2.1 to
3.2.2, and I can't get the migrated mailing lists to work.
When I create new mailing lists there is no problem.
Here are the versions of Mailman I run :
mailman 3.2.2
mailman-hyperkitty (1.1.0)
django-mailman3 (1.2.0)
postorius (1.3.2)
mailmanclient (3.2.2)
HyperKitty (1.2.2)
The command line I use the migrate the mailing list:
/opt/mailman_venv/bin/mailman -C /opt/mailman_venv/var/etc/mailman.cfg
import21 mltest(a)test.domain /var/vmail/mig/data/lists/mltest/config.pck
When I send an email to a migrated list, of which I made myself a
member, no mail is sent, and I get the following stack trace in the log:
==> /var/log/mailman/mailman.log <==
Jan 21 15:02:37 2020 (15948) Uncaught runner exception: None
Jan 21 15:02:37 2020 (15948) Traceback (most recent call last):
File
"/opt/mailman_venv/lib64/python3.6/site-packages/mailman/core/runner.py",
line 173, in _one_iteration
self._process_one_file(msg, msgdata)
File
"/opt/mailman_venv/lib64/python3.6/site-packages/mailman/core/runner.py",
line 266, in _process_one_file
keepqueued = self._dispose(mlist, msg, msgdata)
File
"/opt/mailman_venv/lib64/python3.6/site-packages/mailman/runners/incoming.py",
line 79, in _dispose
process(mlist, msg, msgdata, start_chain)
File
"/opt/mailman_venv/lib64/python3.6/site-packages/mailman/core/chains.py",
line 40, in process
chain = config.chains[start_chain]
KeyError: None
Jan 21 15:02:37 2020 (15948) SHUNTING:
1579615357.5331464+c5918e305ec281d47ef0caaed070c8c4bde26ee3
It seems like the start_chain has value 'None' and that prevents the
mail from processed, but I can't find where to correctly set the
start_chain.
Please let me know how I can remedy this issue.
Kind regards, Paul Honig
This message is intended only for the recipient(s) named above. It may contain proprietary information and/or protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received this e-mail in error, please notify the sender immediately. ESA applies appropriate organisational measures to protect personal data, in case of data privacy queries, please contact the ESA Data Protection Officer (dpo(a)esa.int)
6 years
Re: New year releases!
by Odhiambo Washington
Funny thing is that if you take your time and go over the changelog as
posted you end up nipping the issues in the bud. It just happens that we
are not thorough.
Breaking changes are always documented.
Many years ago I used to go through changelog with a toothcomb, even to see
what new features were there. I'm heading back there.
I was also bitten by the same wasps that bit you, because the release
announcement did not lay bare these issues in the mail. They obviously
expected us to 'behave' like people running critical infrastructure 😂
On Sun, Feb 12, 2023, 17:16 Torge Riedel <torgeriedel(a)gmx.de> wrote:
> Dears,
>
> I like to know and discuss ideas to get the update process more stable!
>
> Today I updated core and web of mailman3 and ran into all the issues you
> can have:
> - Dependencies are not updated properly
> - In case you use "--force-reinstall" Django gets also updated in web and
> you need to update the urls.py file accordingly
> - Enhancement of settings file required (DEFAULT_AUTO_FIELD = ...)
>
> Finally I got it working, but there is always the fear for breaking
> something on update.
>
> I know it is open source and no one gets paid for the work. But I think
> there is space for improvement here.
>
> Kind regards
> Torge
>
> Am 04.01.23 um 10:59 schrieb Abhilash Raj:
> > Hello Everyone,
> >
> > Happy new year!
> >
> > I am pleased to announce several new releases for GNU Mailman project.
> >
> > 1. Mailman Core 3.3.8 [1]
> > 2. Postorius 1.3.8 [2]
> > 3. Hyperkitty 1.3.7 [3]
> > 4. Mailmanclient 3.3.5 [4]
> > 5. Django-mailman3 1.3.9 [5]
> > 6. Mailman-hyperkitty 1.2.1 [6]
> >
> > All the releases are published to PyPI and you can find links to all the
> PyPI projects:
> >
> > https://docs.mailman3.org/en/latest/
> >
> > A full list of change logs are available below:
> >
> > [1]:
> https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.…
> > [2]:
> https://docs.mailman3.org/projects/postorius/en/latest/news.html#news-1-3-8
> > [3]:
> https://docs.mailman3.org/projects/hyperkitty/en/latest/news.html#news-1-3-7
> > [4]:
> https://docs.mailman3.org/projects/mailmanclient/en/latest/src/mailmanclien…
> > [5]: https://gitlab.com/mailman/django-mailman3/-/blob/master/README.rst
> > [6]:
> https://gitlab.com/mailman/mailman-hyperkitty/-/blob/master/README.rst
> >
> > Finally, the minimum Python version required is 3.7, and all the
> components now support Python 3.7 to 3.11.
> >
> > Thanks to all the contributors for making this release possible!
> >
> > If you find any issues, you can report them at the respective bug
> tracker for the projects. The links to issue trackers are available
> > here: https://docs.mailman3.org/en/latest/
> >
> > Installation documentation is available at
> >
> > https://docs.mailman3.org/en/latest/install/install.html
> >
> > and if you are upgrading an existing virtualenv install you can refer
> >
> > https://docs.mailman3.org/en/latest/upgrade-3.2.html
> >
> >
> >
> > _______________________________________________
> > Mailman-users mailing list -- mailman-users(a)mailman3.org
> > To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> > Archived at:
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
> >
> > This message sent to torgeriedel(a)gmx.de
>
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> Archived at:
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>
> This message sent to odhiambo(a)gmail.com
>
2 years, 11 months
Re: About to Install MM3 -- Seeking clarification
by Onyeibo Oku
On Thu, 8 Dec 2022 17:27:13 +0100
Onyeibo Oku <onyeibo(a)schemefusion.com> wrote:
> [Reviving this thread].
>
> I abandoned the hyperkitty setup for a while and returned to it again
> today. I made progress. It turns out that I needed the
> uwsgi-logger-file rpm package and It wasn't installed with the uwsgi
> server. I adapted the settings also, since I was using Emperor mode
> (recall that there are other Django projects on the box). Currently,
> mailman, uwsgi and nginx are all running. They all seem to be happy
> until I assess the mailman-web home URL (e.g. http://my.server.com)
>
> The outcome:
> "502 Bad Gateway"
>
> Nothing shows in the logs except at /var/log/nginx/error.log:
>
> 2022/12/08 15:35:10 [error] 5879#5879: *7 upstream prematurely closed
> connection while reading response header from upstream, client:
> 105.113.33.89, server: my.server.com, request: "GET /favicon.ico
> HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host:
> "my.server.com", referrer: "https://my.server.com/".
>
> Something seems to be wrong between uwsgi and mailman, but I don't
> know where to look.
>
> Regards Onyeibo
Happy New Year to all the Mailman People here (nice rhyme).
I pulled the plugs on uwsgi and installed Gunicorn. Gunicorn gave more
mileage but something is still not right. When I visit the mailmanweb
home with a browser, I see the following:
"Server Error. An error occurred while processing your request."
Mailmanweb and mailman3 services are running. I perused the log file
at /opt/mailman/web/logs/mailmanweb.log and found the following:
raise MailmanConnectionError(
mailmanclient.restbase.connection.MailmanConnectionError: ('Could not
connect to Mailman API: ',
'ConnectionError(MaxRetryError("HTTPConnectionPool(host=\'localhost\',
port=8001): Max retries exceeded with url:
/3.1/lists?advertised=true&count=0&page=1 (Caused by
NewConnectionError(\'<urllib3.connection.HTTPConnection object at
0x7f7f4b128c40>: Failed to establish a new connection: [Errno 111]
0x7f7f4b128c40>Connection refused\'))"))')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/core/handlers/exception.py",
line 55, in inner response = get_response(request) File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/core/handlers/base.py",
line 199, in _get_response response =
self.process_exception_by_middleware(e, request) File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/core/handlers/base.py",
line 362, in process_exception_by_middleware response =
middleware_method(request, exception) File
"/opt/mailman/venv/lib64/python3.10/site-packages/postorius/middleware.py",
line 48, in process_exception return utils.render_api_error(request)
File
"/opt/mailman/venv/lib64/python3.10/site-packages/postorius/utils.py",
line 36, in render_api_error return render(request,
'postorius/errors/generic.html', File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/shortcuts.py",
line 24, in render content = loader.render_to_string(template_name,
context, request, using=using) File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/template/loader.py",
line 62, in render_to_string return template.render(context, request)
File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/template/backends/django.py",
line 62, in render return self.template.render(context) File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/template/base.py",
line 173, in render with context.bind_template(self): File
"/usr/lib64/python3.10/contextlib.py", line 135, in __enter__ return
next(self.gen) File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/template/context.py",
line 254, in bind_template updates.update(processor(self.request)) File
"/opt/mailman/venv/lib64/python3.10/site-packages/django_mailman3/context_processors.py",
line 32, in common context["site_name"] =
get_current_site(request).name File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/contrib/sites/shortcuts.py",
line 16, in get_current_site return Site.objects.get_current(request)
File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/contrib/sites/models.py",
line 59, in get_current return self._get_site_by_id(site_id) File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/contrib/sites/models.py",
line 30, in _get_site_by_id site = self.get(pk=site_id) File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/db/models/manager.py",
line 85, in manager_method return getattr(self.get_queryset(),
name)(*args, **kwargs) File
"/opt/mailman/venv/lib64/python3.10/site-packages/django/db/models/query.py",
line 650, in get raise self.model.DoesNotExist(
django.contrib.sites.models.Site.DoesNotExist: Site matching query does
not exist.
Can someone point me in the right direction please?
Regards
Onyeibo
3 years
Re: Importing config and archives only partially succeeded?
by Eric Broens
Thanks for your prompt reply.To import the config and archives I ran the commands as outlined in Migrating from Mailman 2.1 to Mailman 3 — Mailman Suite 3.3 documentation :
mailman import21 foo-list(a)example.com /path/to/mailman2/foo-list/config.pck
python manage.py hyperkitty_import -l foo-list(a)example.com $var_prefix/archives/private/foo-list.mbox/foo-list.mbox
After running these commands the archives became available for most of the mailinglists. I didn't run the Django admin command with arguments "update_index_one_list" to build the indices for each mailinglist seperately.
After everything was imported, and I noticed that the mails were not shown in the archives for half of the mailinglists, I ran "manage.py update_index" to index the mails for all mailinglists.This command is still running for hours.Since it builds the search index, I don't expect that this would make the mails available on the webpage.
Best regards,Eric
On Sunday, December 13, 2020, 12:12:46 AM GMT+1, Mark Sapiro <mark(a)msapiro.net> wrote:
On 12/12/20 2:33 PM, Eric Broens via Mailman-users wrote:
> Today I have migrated about 30 mailinglists from mailman 2.1 to mailman 3.3.2 & Hyperkitty 1.3.3.
> Although no error responses were given when importing the 2.1 pck files, and neither for importing the archives, for about half of the mailinglists the hyperkitty webpage doesn't show any subscribers, nor archived emails.
> The imported members and other settings are correctly shown in the postorius page.
> Is there anything that I can verify why the archive is not shown on the hyperkitty page?The hyperkitty_email database table does contain the emails.
Did you specify --no-sync-mailman when you ran hyperkitty_import? If so
did you run the django sync_mailman job when you finished importing?
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
_______________________________________________
Mailman-users mailing list -- mailman-users(a)mailman3.org
To unsubscribe send an email to mailman-users-leave(a)mailman3.org
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
5 years, 1 month
Re: Setting for archiving multi email domains?
by Duane Raymond
Thanks!
So - I've documented what I did below. Now that I am getting bouces, I got an email that said "The response from the remote server was:
550 5.1.1 <test(a)emaildomainA.com>: Recipient address rejected: User unknown in virtual alias table"
Specifically to some ot the questions above:
> What do you mean by "set"? That the list has archiving enabled?
Yes. I meant that archiving had been enabled for the test(a)emaildomainA.com test list.
Shunting error: I think the shunting error is a distraction if it is caused by not having a recipient. Early tests to the test list had me as an administrator, but not as a member - so this would have caused that. Once I added myself (before I sent the original message in this thread), the result (not sending the message to the list using a 'virtual' domain and not archiving it) continued.
Virtual Domains:
So based on the Postfix virtual domain documentation and the Mailman3 specific "Unusual Postfix configuration' section, I added the following to my postfix main.cf file:
virtual_alias_domains = emaillistA.com emaillistB.com
virtual_alias_maps = hash:/opt/mailman/mm/var/data/postfix_vmap
and removed those domains from the 'mydestination' setting.
I then created the postfix_vmap file and added to it:
emaillistA.com DOMAIN
#
emaillistB.com DOMAIN
...then ran postmap with the path and reloaded postfix.
On testing I also noticed that a previous counce was to emaildomainC.com - an alias for emaildomainA.com I had originally setup in the Postorius domain setup and then removed. I guessed that it was still in the postfix config, so I double checked in the MM generated postfix_domains file and ran postmap /path-to/postfix_domains and reloaded postfix again...and sent another test message....
This then generated the error email at the top from my email client (gmail).
...but the end result is no email sent and thus the reason there is no archive....
Any further suggestions? I can supply other log messages and configuration options if it helps.
Duane
4 years, 10 months
Re: Postorius is not working (throws "Error 500" suddenly after changing POSTORIUS_TEMPLATE_BASE_URL and changing back)
by Stephen J. Turnbull
Wikinaut writes:
> I fully understand. Need to look up how to smoothly backup the
> mailman3 settings and list databases in order to make that move
> with losing stuff... (kinda "mailman3 backup all" and mailman3
> restore all" :-)
One piece of generic Python advice: don't mix OS packages and pip
(except for Python itself). Always use a venv if you need to use pip
for packages that aren't in your OS distro. (Of course it's possible
to do so in many cases if you *really* know what you're doing, but
it's almost never worth the pain.)
All of the site configuration data should be in /etc/mailman3 (except
for a few template files in $var_dir/templates), and all of the
domain, list configuration, and archive data is in the RDBMS (usually
PostgreSQL or MySQL). The RDMBS should be backed up regularly
according to its procedures, but that's completely out of scope for
Mailman.[1] The RDBMS should "just work" with exactly the same
Mailman-side configuration as Debian.[2]
As I think Odhiambo mentioned, the Mailman side of the site
configuration also does not change (for example, the RDBMS URL and
authentication). What you do want to change is the file system
configuration. Except for /etc/mailman3, it should be completely
different -- as with pip'ing packages, you don't want to share files
across a local venv and the OS version of the same package. It just
leads to pain.
I think among the developers the preferred root = /opt/mailman (or
/opt/mailman3) for Mailman's directory hierarchy. This will be
populated with subdirectories when you run `mailman info` (or start
Mailman) the first time. The directories relevant to configuration
are all under $root/var, and you inform Mailman about this by setting
`var_dir` in mailman.cfg. If there are other file and directory
configurations in mailman.cfg, they need to changed from the Debian
values (probably /var/lib/mailman3/...). You may do fine by simply
deleting all other directory configurations and let Mailman decide
where to put them relative to the `var_dir`,
You will need to fix the setting for STATIC_ROOT (I think that's the
name, if not search for "static") in settings.py, and maybe in
mailman-web.cfg. While it's not necessary (and may be a little
inconvenient), I recommend changing the root for Mailman's log files
from the Debian value (probably /var/log/mailman3) to the Mailman
default of $root/var/logs, and similarly for the Django logs in
settings.py.
Non-Mailman settings
I think you will need to change the settings for your MTA (usually
Postfix or Exim4) so that it will recognize Mailman lists and route
them to Mailman correctly. There will probably be mailman-related
files in /etc/$MTA/conf.d (or something like that). You can edit them
directly, but I recommend changing the name (paranoia so that dpkg
doesn't overwrite them).
The only thing I can think of for the webserver (Apache?) is the
location of the static directory. I think Debian uses uwsgi by
default so you may need to adjust its configuration too.
I think you need to fix logrotate's configuration in
/etc/logrotate.d. Not sure how many mailman-related config files are
in there.
I think you mentioned systemd .service files. If so that doesn't make
sense to me -- Debian should provide those. You need to make sure
those are disabled or deleted so that Debian Mailman services don't
start on reboot. This is automatic if you delete the Debian packages,
of course, but not for any files you created yourself. Probably the
Debian .service files will work, but you need to make sure they refer
to the venv Mailman (using the full path to the `mailman` utility in
the venv's bin directory will work, and similarly for `mailman-web`).
systemd does not configure the /opt hierarchy in it search path by
default. I put the mailman, qcluster, and gunicorn (if you use that
instead of uwsgi) .service files in /usr/local/lib/systemd/system, and
that works for me.
Flotsam and jetsam
If you have run Mailman with any success at all, you may find some
stuff in Debian's $var_dir/queue/* directories or in its
$var_dir/lists/* directories. The queue/* directories contain
incoming messages at various stages of processing, and the lists/*
directories contain files containing multiple messages that are
scheduled to be sent out as digests. You can read the queue files
with `mailman qfile /path/to/queue/file.pck`, and the digests are just
text files. That way you can check if there's anything important in
them. If not, you can just delete them. If there is, you can just
copy them to the same place in the venv $var_dir as they were in
Debian's $var_dir, but you may want to ask here first before doing
that. It's best to let Mailman run until all the queues are cleared
(except maybe bad and shunt, but if those have any files in them you
need to intervene anyway).
Afterword
I'm sure this sounds complicated, but in fact it's a pretty short list.
When you get there we'll help you.
Footnotes:
[1] But not for this list. You may as well ask here, somebody
probably will tell you the basics quickly. But for troubleshooting,
go the the RDBMS's own channels.
[2] In fact, it is possible to keep the Debian installation, copy the
the mailman.cfg somewhere, edit it, stop Debian Mailman, and start up
the venv Mailman. I don't recommend this -- the point is that the
venv and the databases are that isolated from the rest of the system.
2 years, 3 months