Search results for "Postorius" - Mailman-users

Re: mailman import21

by Brian Carpenter

On 10/11/20 11:06 AM, Mark Sapiro wrote: > On 10/11/20 7:41 AM, Brian Carpenter wrote: >> Does the import21 script also import custom templates? > No. > > >> Are these custom >> templates contained in the config.pck file? > No. They are in the same MM 2.1 directory as the config.pck, but in a > sub-directory for the language code, i.e. lists/LISTNAME/LC/ > > >> Are there any known issues >> with the importing of mm2 templates into a mm3 environment? > > The replacements are different. i.e., things like %(email)s and > %(listname)s become $user_email and $short_listname. > > Also, there are some MM 2.1 templates that have no MM 3 equivalent. > Also, things like message and digest headers and footers that are MM 2.1 > list attributes that are templates in MM 3, although import21 tries to > handle those. So are how these templates showing up on my Mailman 3 servers? They show up in /opt/mailman/mm/var/templates/lists/listname/en/. I have a client whose old footer template from Mailman 2 still showing up in list messages. He then created a custom footer template via Affinity (same thing happen in Postorius) and both custom footers were being included in list messages. -- Brian Carpenter Harmonylists.com Emwd.com

6 years

MAILMAN_ARCHIVER_KEY - 401 error

by Torsten R.

Hi together, I just updated one of my servers to Debian 11 and that also introduced mailman3 to the system as I was running mailman 2.1 for years before on that server. It took me a while to configure everything but I am convinced that I now have everything done properly. I am able to access the web interface of mailman3 (postorius) and also the archive (hyperkitty). I configured exim to deal with messages send to the lists. But when I try to send a message (as a member of a list) to a list I always get a 401 in mailman.log and the message that I should check if I provided MAILMAN_ARCHIVER_KEY and if it is correct. So far, I can say that the key is the same in mailman-web.py (with single quotes) and mailman-hyperkitty.cfg (without quotes). in mailman-web.py I have: MAILMAN_ARCHIVER_KEY = 'somesecretkeywithlotsofcharacters' and in mailman-hyperkitty I have this: api_key: somesecretkeywithlotsofcharacters I have some messages in $var_dir/archives/hyperkitty/spool but mailman qfile <messagefile> doesn't show anything that would catch my eye. So, maybe I need another pair of eyes that could take a look at my configs or someone that is turning me in the right direction and might give some hints where to look and what to check. Thanks, Torsten

4 years, 6 months

Re: Hyperkitty displaying example.com

by Jeff

You probably need to adjust the SITE_ID in your settings.py file. Your current SITE_ID is probably 1 and you just need to make it 2. > On Sep 21, 2022, at 8:58 PM, Mark Dale via Mailman-users <mailman-users(a)mailman3.org> wrote: > > Hi > > On a new installation I'm seeing "example.com" displayed at the top of > the HyperKitty pages. > > I was expecting to see "lists.mydomainname.com", so I think I've missed > something somewhere. > > Also, when a user signs up, the confirmation email reads as: > > ////////////////////////// > > SUBJECT: [example.com] Please Confirm Your E-mail Address > FROM: postorius(a)lists.mydomainname.com > > BODY: > Hello from example.com! > > You're receiving this e-mail because user yogibear has given your e- > mail address to register an account on example.com. > > To confirm this is correct, go to > https://lists.mydomainname.com/mailman3/accounts/confirm-email/NQ:1obAI3:Yw… > > Thank you for using example.com! > example.com > > ////////////////////////// > > Any clues greatly appreciated. > > Thanks, > Mark > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

4 years, 1 month

Re: My list has been switched from public to private archives, but they are still visible

by Abhilash Raj

On Thu, Sep 19, 2019, at 3:31 AM, Paul Arenson wrote: > Can anyone help with this? > > I set my list to private archives today. Up until then, it was pubic. > And yet, if you go to the list page, you will see that if you click > archives (top left), you can see them. (Filled with a bunch of tests > posts for now) This change takes a bit of time to sync up Hyperkitty. THere is a daily cron job which syncs this information from Mailman Core. I have made a change which should fix this problem and the process would become synchronous. You can run `python manage.py runjobs --daily` command to force sync that information. > > https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres… > > Am I misunderstanding what a private archive is? Not really, your expectation is right. > > Is something wrong with my setup (which is hosted)? > > Thank you. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

7 years, 1 month

Re: MM3 Postfix FROM header issue

by Mark Sapiro

On 12/17/23 12:17 PM, eboltz(a)lhtservices.com wrote: > Mark, > > Thanks for the assistance thus far. > > When I create a new Domain within Postorius. If I set the Mail Host to 'example.com' and the Alias Domain to 'lists.example.com', create a new list such as 'staff_test', it'll default to the 'example.com' domain (which is the real domain). Then when I subscribe, I'll get an email from 'staff_test(a)example.com', when I want it to be from 'staff_test(a)lists.example.com' instead, also if I reply back to the 'staff_test(a)example.com' then I"ll get a bounce-back that it doesn't exist, but if I send to 'staff_test(a)lists.example.com' it'll go to the newly created list. I'd like all defaults to be '@lists.example.com', if that makes sense. You want to create the list with Mail Host lists.example.com and no Alias domain. Then in Postfix you want no virtual_alias_maps setting. The rest of what you show for postconf -n at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… is good. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 10 months

Apache Configuration for Subdomain

by Stephen J. Turnbull

Mail Person writes: > I'm trying to move my mailman3 web interface from > example.com/mailman3 to just lists.example.com like this site. I am > not having luck with the apache configuration. I've tried below but > I just get a 403: Apache typically runs as www-data or some such user, while Mailman resources are typically owned by mailman, lists, or list (depending on distribution and administrator whim). Does the webserver have the appropriate permissions to access /var/lib/mailman3/web/static? Perhaps when uwsgi forks it changes effective uid, in which case it needs to have access. Mark may have better ideas. > > <VirtualHost *:80> > > ServerAdmin admin(a)example.com > ServerName lists.example.com > ServerAlias www.lists.example.com > > > <Directory "/var/lib/mailman3/web/static"> > Require all granted > </Directory> > > <IfModule mod_proxy_uwsgi.c> > ProxyPass /favicon.ico ! > ProxyPass /static ! > ProxyPass / unix:/run/mailman3-web/uwsgi.sock|uwsgi://localhost > </IfModule> > > </VirtualHost> > > > Alias /favicon.ico /var/lib/mailman3/web/static/postorius/img/favicon.ico > Alias /static /var/lib/mailman3/web/static > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > >

4 years, 3 months

Re: UnpicklingError on mailman lists command

by Bryan Loniewski

Okay, so the issue has been fixed by updating the data to an empty list equivalent (after finding, confirming and copying another lists empty value): MySQL [mailman]> SELECT accept_these_nonmembers INTO @newatn FROM mailinglist WHERE list_name='foo_is_a_good_list_with_emptyval'; MySQL [mailman]> UPDATE mailinglist SET accept_these_nonmembers=@newatn WHERE list_name='the_borked_list'; Query OK, 1 row affected (0.02 sec) Rows matched: 1 Changed: 1 Warnings: 0 Thanks for the help on troubleshooting that! Based on your import21 comments, I'm realizing the issue may be caused by multiple runs of import21. I have been (although I've stopped it for some time now) running import21 nightly to sync up my Mailman3 list data with Mailman2. I've been doing the same for archiving as well. I've noticed now that some lists have duplicate acceptable alias entries (looks to be like <number of aliases> x <number of import21 runs>) and the same goes for *_these_nonmembers entries. Is this a symptom of running the old(er) import21 command pre-3.3.2? Will the 3.3.2 import21 work as expected w/out the duplications if I was to clear out the appropriate lists col data (i.e., is it okay to run import21 multiple times)? I tried to figure out if there were any other duplicative data, but did not come across any. Is there more that I'm missing? ________________________ Bryan Loniewski Rutgers University OIT:EAS:Messaging and Collaboration Services System Administrator ________________________________________ From: Mark Sapiro <mark(a)msapiro.net> Sent: Tuesday, August 3, 2021 12:28 PM To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: UnpicklingError on mailman lists command On 8/3/21 7:35 AM, Bryan Loniewski wrote: > Okay, that helped identify possible culprits. Three of our lists are hitting/over the max BLOB length (65,535 bytes) for accept_these_nonmembers. I executed this SQL command for each of the BLOB cols you referenced and only accept_these_nonmembers is borked: > > MySQL [mailman]> SELECT list_name, OCTET_LENGTH(accept_these_nonmembers) as howManyBytes from mailinglist where OCTET_LENGTH(accept_these_nonmembers) > 64000; > +---------------+--------------+ > | list_name | howManyBytes | > +---------------+--------------+ > | <snip> | 65535 | > | <snip> | 65535 | > | <snip> | 65535 | > +---------------+--------------+ > 3 rows in set (0.01 sec) > > So, putting aside *why* these lists are so large wrt that column, can or should I alter the col type(s) to MEDIUMBLOB? If yes, would that remain on suite upgrades? If no, is there a better alternative? Or some recommendation in between? Modifying the column type at this point will not help because the data has already been truncated and the missing piece is gone. It may help avoid future issues, but I suspect it's not necessary. It would probably 'stick' on an upgrade as long as there were no new migration affecting those columns, but if there were such a migration, it could fail due to the existing type not being as expected. I suspect these lists were imported from Mailman 2.1 with the `mailman import21` command. The goal of that is to import the addresses as non-members with the appropriate action and put only the regexps in the MM 3 *_these_nonmembers. Prior to Mailman 3.3.2, import21 would add the entire 2.1 *_these_nonmembers and then delete the non-regexps. Possibly something went wrong in that process because of the truncation and left the new *_these_nonmembers in a broken state. Or possibly you just have a huge number of regexps in these list's accept_these_nonmembers. I suggest you use mysql to update this column's data for these three lists. to a value which unpickles as an empty list. This value is the python bytes b'\x80\x04\x5d\x94\x2e' (i.e. 5 bytes with hex values 80, 04, 5d, 94 and 2e). I'm not sure how that is stored in a MySQL BLOB, but I'm sure you can determine that. This will allow you to actually access those lists in Mailman. Also note that beginning with Postorius 1.3.3, the *_these_nonmembers settings are exposed on the List Settings -> Message Acceptance form. If you still have the original MM 2.1 config.pck files, you can examine them with Mailman 2.1's bin/dumpdb command to see what was in accept_these_nonmembers, and then see if the appropriate MM 3 nonmembers were created and if the remaining regexps are not too long, add them. If you need help with this, I will help. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mai…

5 years, 2 months

Re: How to delete an archive

by Abhilash Raj

On Wed, Sep 4, 2019, at 6:06 PM, Andrew Bernard wrote: > I deleted a list but I want to delete the archive as well. > > How do you do that? I know this is a big pain point for many people, but this doesn't work right now on MySQL, which is why I haven't been able to commit my fix for this yet :( > > Also, I had a list called Test. I deleted it. I made a new list called > Testing, with a different email address, bu the archive points to the > test list. This seems to be messed up. Why? I am sorry to say I have > had continuous trouble with many aspects of Hyperkitty from the > beginning several months ago. I know that is not helpful, but I am > starting to wonder about all this. Deleting a list doesn't delete the archives. This the default behavior and perhaps the user interface can be improved by optionally allowing deletion of archives. I created: https://gitlab.com/mailman/postorius/issues/382 > > Andrew > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

7 years, 1 month

Re: Using REST with CLI using curl .... help needed

by Stephen J. Turnbull

kontakt(a)michaelbakonyi.de writes: > I have the same usecase as Krešimir and tried the example of > Mark. So in mailman.cfg I set hostname to > "https://lists.mydomain.org" and then fire this command I don't recall this, and unfortunately Mark is on vacation and completely offline until mid-September, except for maybe a couple days next week. > curl -urestadmin:mypw --referer https://lists.mydomain.org --request POST -H 'Content-Type: application/json' --data-binary '{"list_id": "thelist.lists.mydomain.org", "subscriber": "user(a)example.net", "display_name": "Joe User", "pre_verified": true, "pre_confirmed": true, "pre_approved": true}' https://lists.mydomain.org/mailman3/postorius/lists/thelist.lists.mydomain.… > > But then I get a CSRF-warning. Do I now have to get the token and > put it in my POST or is there an easier way to do this? As far as *I* know there's no way to avoid the CSRF warning. If that means you can't get the response data you need, you will have to do the CSRF two-step. With luck maybe Mark will see this in a few days but I think he's going to be busy preparing the 2d half of his trip. Regards, Steve

4 years, 2 months

Re: Hyperkitty performance problem

by Greg Newby

Hi monochromec. I have a similar installation and was also having various problems and delays. I'm on a similarly sized system (Ubuntu 24.04 with venv installations of mailman3; 32GB memory). What I found was that tuning of the resources made a very significant impact. Out of the box tuning parameters for the various components were not sufficient to keep mail flowing. The "system info" tab in the web interface tells you where things are held up, if they are. I was having trouble with the 'out' queue getting slow, and the postfix 'mailq' was showing thousands of messages waiting to be delivered. I was running out of smtpd processes. Archiving was delayed (though not as much as yours). Some of the things I did: - Increase PostgreSQL resource limits. Some of the parameters in my configuration files: # Tuning, via https://pgtune.leopard.in.ua/ for 'data warehouse' app. max_connections = 100 # default shared_buffers = 8GB effective_cache_size = 24GB maintenance_work_mem = 2GB # wal_buffers = 16MB default_statistics_target = 500 effective_io_concurrency = 2 work_mem = 26214kB min_wal_size = 4GB max_wal_size = 16GB max_parallel_workers_per_gather = 4 max_parallel_maintenance_workers = 4 - Increase the number of mailman processes in mailman.cfg # Note: Values need to be a power of 2. Make sure PostgreSQL allows more connections than the # of instances [runner.in] class: mailman.runners.incoming.IncomingRunner instances: 16 # instances: 8 [runner.out] class: mailman.runners.outgoing.OutgoingRunner instances: 64 The "out" setting, notably, decreased mail delivery times. - Increase postfix resources. Notably in main.cf: # https://www.postfix.org/TUNING_README.html # default_process_limit = 100 # default default_process_limit = 250 Maybe these will point you in the right direction. I realize you are focused on hyperkitty, and my experience was that all the major components (database, MTA and mailman3) all needed more resources. ~ Greg On Sat, Dec 28, 2024 at 7:26 AM monochromec via Mailman-users < mailman-users(a)mailman3.org> wrote: > Following the root cause analysis which Tobias started all those weeks ago > we (the admin team behind the installation) are still struggling with the > following phenomenon: messages on average take more than 24 hours to be > processed, more precisely, the average lifetime of a pickled message object > in `/var/lib/mailman3/queue/pipeline` clocks in at around 26 hours). > > Couple of stats of the installation: standard installation from Bookworm > OS repos, Hyperkitty as archiver and Postorius as web frontend as explained > above, running Python 3.11.2 from the standard systemd service as packaged > with the Bookworm deb file. All backends (Core + Hyperkitty) are supported > by Postgres version 15+248. The MTA is a standard Postfix installation, > again from OS packages. > > The underlying VM has 7 cores with just under 24 GB of main memory. This > production instance is handling less than 130 mailing list (MLs) with an > average of less than 10 postings per day per ML. CPU core utilisation > hovers around 50% with the lion share allocated to the four pipeline > runners as part of the MM configuration. > > OS resource utilisation is well below bounds (approx. 8 GB of main memory > allocated to running processes), plenty of available sockets space (I > noticed some transient `Connection lost during _handle_client()` warnings > in the logs so I checked that the SMTP runner can connect to Postfix for > delivering the messages after processing by checked the socket allocation > of the running processes). > > Cursory review of the corresponding Core classes (runner + pipeline > implementation in addition to `posting_pipeline`) didn't reveal any further > pointers. What I did notice though that increasing the logging levels of > the components (namely `smtp`, `pipeline` and `lmtp` to `debug`) in > `/etc/mailman3/mailman.cfg` didn't add any useful information to the logs > as configured after restarting the Core. > > As outlined above, Hyperkitty doesn't seem to do a check based on ML and > message ID before archiving a message in the database. But this only add a > REST roundtrip and Postgres communication through Hyperkitty's underlying > Django framework to the overall system load and the driving UWSGI instance > is well within CPU cycle bounds. > > Any pointers are appreciated - more than happy to provide more info if > required. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to gbnewby(a)petascale.org >

1 year, 10 months

Mailman-users search results for query "Postorius"