Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: Held messages: Long time waiating for response of the Mailman API

by Stephan Krinetzki

Mark Sapiro wrote: > On 1/29/22 05:48, Jacob Sievert via Mailman-users wrote: > > Hello, > > we seem to have the same problem right now. > > We are still on python3.6 and postgresql 12.9 > > I looked at our Table pendedkeyvalue and we have roughly 35k rows in > > that table, > > is that normal or shouldn't those get cleaned up after a while? > > Yes they should. This is https://gitlab.com/mailman/mailman/-/issues/257 > fixed in Mailman core 3.3.5. Also in Mailman 3.3.5 is a new Task runner > that runs periodic tasks, one of which is to remove orphaned pendings. > This may also be the issue for the OP in this thread if that > pendedkeyvalue table is also large. I can confirm this. We have a total of ~580000 Rows in the pendedkeyvalue table. Since last wednsday even the hourly job quits (Out of memory exception) > Here is a mailman shell script that will clean that up. > # Prior to Mailman 3.3.5, some tokens for user confirmations were pended > with > # too long a lifetime. This script removes those pendings based on when > they > # were pended and the configured pending_request_life rather than their > # expiration. > > # Also prior to Mailman 3.3.5, pended held_message tokens for email handling > # of the message were not removed when the message was handled via REST. > This > # script removes those pendings too. > > # This is run with > # mailman shell -r delete_orphans_expireds > # after saving it as > # /opt/mailman/mm/venv/bin/delete_orphans_expireds.py > > from datetime import datetime > from lazr.config import as_timedelta > from mailman.config import config > from mailman.database.transaction import transactional > from mailman.interfaces.pending import IPendings > from zope.component import getUtility > > pendings = getUtility(IPendings) > > def is_request(id): > if config.db.store.execute( > 'SELECT * FROM _request WHERE id = {};'.format(id)).rowcount > 0: > return True > return False > > then = datetime.now() - as_timedelta(config.mailman.pending_request_life) > thenm = datetime.now() - as_timedelta(config.mailman.moderator_request_life) > > def fromisoformat(x): > if hasattr(datetime, 'fromisoformat'): > return datetime.fromisoformat(x) > try: > return datetime.strptime(x, '%Y-%m-%dT%H:%M:%S.%f') > except ValueError: > return datetime.strptime(x, '%Y-%m-%dT%H:%M:%S') > > @transactional > def delete_orphans_expireds(): > count = 0 > for token, data in pendings.find(pend_type='held message'): > if data and not is_request(data['id']): > result = pendings.confirm(token, expunge=True) > count += 1 > print(f'expunged {count} orphaned pended held messages') > > count = 0 > for token, data in pendings.find(pend_type='data'): > if data and data['_mod_hold_date']: > when = data['_mod_hold_date'] > if isinstance(when, str): > when = fromisoformat(when) > if when < thenm: > result = pendings.confirm(token, expunge=True) > count += 1 > print(f'expunged {count} expired held messages') > > pends = list(pendings.find(pend_type='subscription')) > pends += list(pendings.find(pend_type='unsubscription')) > count = 0 > for token, values in pends: > if values and values['token_owner'] == 'subscriber': > when = values['when'] > if isinstance(when, str): > when = fromisoformat(when) > if when < then: > result = pendings.confirm(token, expunge=True) > count += 1 > print(f'expunged {count} expired (un)subscription confirmations') > > The above says to save the script at > /opt/mailman/mm/venv/bin/delete_orphans_expireds.py but that path may > need to be adjusted based in where Mailman's bin/ directory is in your > installation. Thanks for the Script Mark! WIth the Script, the result is: mailman shell -r delete_orphans_expireds Jan 31 09:15:46 2022 (7980) Database url: postgres://mailman:XXXXXXXX@127.0.0.1/YYYYYYY expunged 95918 orphaned pended held messages expunged 7958 expired held messages expunged 22372 expired (un)subscription confirmations And the pendedkeyvalue table has significant lower entries (~ 92000) and now the process to accept or decline a held message ist a lot of faster. Maybe we will take the database to a seperate server to get more speed - but for the moment it is fast enough. Thanks again and thanks to Jacob Sievert for the pointer to the size of the pendedkeyvalue table.

3 years, 10 months

[MM3-users] Re: integrating mm3 with postfix / lmtp

by Abhilash Raj

On Nov 13 2017, at 1:34 am, Thor Atle Rustad <thor.rustad(a)gmail.com> wrote: > There is a way around it! > > I have had two issues with the the maxking docker image. One is that the regexp is not working properly. I reported that, and it has been fixed in newer code. My other problem is that the docker image creates a user, mailman, that receives uid 103. Well, uid 103 on my system is already taken by systemd-bus-proxy (grep 103 /etc/passwd returns "systemd-bus-proxy:x:103:105:systemd Bus Proxy,,,:/run/systemd:/bin/false"). Containers usually run with uid namespace, so it doesn't really matter what uid is used outside of container. Unless, you mount /etc/shadow from host in to the container, which isn't really needed for the images. > > My solution includes downloading the corrected postfix.py, and replacing the Dockerfile. I put the postfix.py in <docker-mailman>/core/assets/. > > My Dockerfile: > FROM maxking/mailman-core > > RUN grep mailman /etc/passwd && grep mailman /etc/group \ > && deluser mailman \ > && addgroup -S -g900 mailman \ > && adduser -S -u900 mailman mailman \ > && grep mailman /etc/passwd && grep mailman /etc/group > COPY assets/postfix.py /usr/local/lib/python3.6/site-packages/mailman/mta/postfix.py > > > I then run docker build (with -t parameter, you must look up that yourself). I use a different name for my images, so I end up with (note, there are two tags per image): > root@mailer:/home/mailman/docker/docker-mailman_mods/core# docker images > REPOSITORY TAG IMAGE ID CREATED SIZE > local/mailman_core_900 20171110_2 9649e84767e1 2 days ago 176MB > local/mailman_core_900 latest 9649e84767e1 2 days ago 176MB > local/mailman_web_900 20171110_2 07a9b3d7ddd6 2 days ago 247MB > local/mailman_web_900 latest 07a9b3d7ddd6 2 days ago 247MB > > > I do the same with the web image, as I need to change the user there, too. > > Then, in docker-compose.yaml, I change the line(s) referring to the image name(s): > > services: > mailman-core: > image: local/mailman_core_900 > container_name: mailman-core > hostname: mailman-core > > > > mailman-web: > image: local/mailman_web_900 > container_name: mailman-web > hostname: mailman-web > > > I don't know if this is a good solution, but at least it fixes some serious issues with the 3.1's postfix integration that wouldn't otherwise be fixed until the 3.2 release. The bottom line is that it works for me, but it adds an additional layer of complication. > > 2017-11-03 19:40 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in (mailto:maxking@asynchronous.in)>: > > On Fri, Nov 3, 2017, at 08:29 AM, Fabian A. Santiago wrote: > > > October 26, 2017 11:07 PM, "Mark Sapiro" <mark(a)msapiro.net (mailto:mark@msapiro.net)> wrote: > > > > > > > On October 26, 2017 7:30:35 PM PDT, "Fabian A. Santiago" <fsantiago(a)garbage-juice.com (mailto:fsantiago@garbage-juice.com)> wrote: > > > > > > > >> That was it. Perfect. I manually modified my regexp map and it works > > > >> now. Excellent and Thank you. You're the man. Does mm3 ever refresh > > > >> those maps or only as I add new domains / lists? > > > > > > > > Only when you make changes to domains or lists. > > > > > > > > -- > > > > Mark Sapiro <mark(a)msapiro.net (mailto:mark@msapiro.net)> > > > > Sent from my Not_an_iThing with standards compliant, open source software. > > > > > > Mark, > > > > > > I've noticed that even simply restarting the mm3 components those alias > > > maps get rewritten and the problem returns until I can manually edit it. > > > > Yeah, that is true. Transport maps are re-generated everytime the > > container restarts. > > > > I don't think think there is any way around this right now :( > > > > > > -- > > Abhilash Raj > > maxking(a)asynchronous.in (mailto:maxking@asynchronous.in) > > _______________________________________________ > > Mailman-users mailing list > > mailman-users(a)mailman3.org (mailto:mailman-users@mailman3.org) > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

8 years, 1 month

[MM3-users] Re: mail loops back to myself

by Florian Sukup

On 6/9/25 05:05, Mark Sapiro wrote: > On 6/8/25 16:04, Florian Sukup wrote: >> >> I set up mailman, created a site/domain (lists.yyy.com) My hostname >> is host.xxx.com . My IP has a RDNS entry: reverse.rrr.com . > > Why is the RDNS to reverse.rrr.com and not host.xxx.com. It is important > for mail delivery to have full circle DNS. I.e. the sending server > should have an A record for its IP and revers DNS for that IP should > point back to the sending server's name. I'm guessing that rrr.com is a > hosting provider and you don't control the rDNS for that IP, but you > should try to get them to change it for you. Without that change, > delivery of your outbound mail, at least to large ISPs, will be > problematic at best. > The setup has historic reasons. However I can eliminate reverse.rrr.com completely and replace it by host.xxx.com. Right now reverse.rrr.com has an A-record pointing to the host's ip address. > >> When I send an email to my mailing list mylist(a)lists.yyy.com I receive >> an error email. The logfile says the following: >> >> ... >> Jun 9 00:23:38 arvak postfix/relay/smtp[]: 094895FADB: >> to=<mylist(a)lists.yyy.com>, relay=reverse.rrr.com[m.y.i.p]:25, >> delay=0.06, delays=0.03/0.01/0.02/0, dsn=5.4.6, status=bounced (mail >> for lists.yyy.com loops back to myself) > > What is the output from `postconf -n`? > alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases append_dot_mydomain = no biff = no compatibility_level = 2 inet_interfaces = all inet_protocols = all local_recipient_maps = proxy:unix:passwd.byname $alias_maps hash:/var/lib/mailman3/data/postfix_lmtp mailbox_size_limit = 0 message_size_limit = 0 mydestination = localhost, localhost.localdomain, arvak myhostname = host.xxx.com mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128, 192.168.77.0/24 myorigin = /etc/mailname owner_request_special = no readme_directory = no recipient_delimiter = + relay_domains = ${{$compatibility_level} < {2} ? {$mydestination} : {}} hash:/var/lib/mailman3/data/postfix_domains relayhost = smtp_tls_CApath = /etc/ssl/certs smtp_tls_cert_file = /etc/letsencrypt/live/host.xxx.com/fullchain.pem smtp_tls_key_file = /etc/letsencrypt/live/host.xxx.com/privkey.pem smtp_tls_loglevel = 3 smtp_tls_security_level = may smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtp_use_tls = yes smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination smtpd_tls_cert_file = /etc/letsencrypt/live/host.xxx.com/fullchain.pem smtpd_tls_key_file = /etc/letsencrypt/live/host.xxx.com/privkey.pem smtpd_tls_loglevel = 3 smtpd_tls_security_level = may smtpd_use_tls = yes transport_maps = hash:/var/lib/mailman3/data/postfix_lmtp virtual_alias_domains = 9 different domains, however not host.xxx.com, lists.yyy.com or reverse.rrr.com virtual_alias_maps = hash:/etc/postfix/virtual >> Jun 9 00:23:38 arvak postfix/smtpd[114119]: disconnect from >> host.xxx.com[m.y.i.p] ehlo=1 quit=1 commands=2 >> ... >> >> The MX record of lists.yyy.com points to reverse.rrr.com. Not sure if >> this is the best idea? > > reverse.rrr.com has no A or AAAA record. An MX MUST point to a domain > that has an A or AAAA record. The MX should point to host.xxx.com. > Will be resolved (s. above). >> Can anyone give me a hint where to search for this error? > > The output from `postconf -n` would help. Also, have you set up postfix > per > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h… ? > Basically yes, but I see a few differences which all worked on my test installation. Here comes my mailman.cfg: [mailman] site_owner: mailman@... noreply_address: noreply default_language: en sender_headers: from from_ reply-to sender email_commands_max_lines: 10 pending_request_life: 3d cache_life: 7d pre_hook: post_hook: layout: debian filtered_messages_are_preservable: no html_to_plain_text_command: /usr/bin/lynx -dump $filename listname_chars: [-_.0-9a-z] [shell] prompt: >>> banner: Welcome to the GNU Mailman shell use_ipython: no history_file: [paths.debian] var_dir: /var/lib/mailman3 queue_dir: $var_dir/queue bin_dir: /usr/lib/mailman3/bin list_data_dir: $var_dir/lists log_dir: /var/log/mailman3 lock_dir: $var_dir/locks data_dir: $var_dir/data cache_dir: $var_dir/cache etc_dir: /etc/mailman3 ext_dir: $var_dir/ext messages_dir: $var_dir/messages archive_dir: $var_dir/archives template_dir: $var_dir/templates pid_file: /run/mailman3/master.pid lock_file: $lock_dir/master.lck [database] class: mailman.database.sqlite.SQLiteDatabase url: sqlite:///$DATA_DIR/mailman.db debug: no [logging.debian] format: %(asctime)s (%(process)d) %(message)s datefmt: %b %d %H:%M:%S %Y propagate: no level: info path: mailman.log [webservice] hostname: localhost port: 8001 use_https: no show_tracebacks: yes api_version: 3.1 admin_user: ... admin_pass: ... [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver smtp_host: localhost smtp_port: 25 smtp_user: smtp_pass: lmtp_host: 127.0.0.1 lmtp_port: 8024 configuration: python:mailman.config.postfix Thanks for your help, Florian.

6 months, 1 week

[MM3-users] Re: Member Issue Discovered

by Brian Carpenter

On 10/20/20 11:19 PM, Mark Sapiro wrote: > On 10/20/20 6:54 AM, Brian Carpenter wrote: >> Respectively, I think you are asking the wrong question here. The real >> question is why isn't a display_name being removed when a list >> subscriber is unsubscribed. > > I'd like to understand the real requirement. It seems to me that this > issue has come up because a list admin wanted to change the display name > shown in the membership roster for a user. Since there is currently no > UI to do this, the list admin tried to do it by unsubscribing and > resubscribing the user. That didn't work which led to this > "unsubscribing a user should remove the user's information" thread, but > the real issue is the lack of a UI for changing display names. It seems > if that UI existed and was available, the "unsubscribing a user should > remove the user's information" issue would never have been raised. There are two real requirements. One is to be able to do something as easy as changing a name for a list member. I did a lot of testing with the relationship between a name used for a subscription versus a name used for registering via the U.I. (Postorius/Django) and it is very confusing. I still am having a very difficult time understanding the logic presented here for the way Mailman 3 handles user information. The second requirement is ALL data should be removed if someone unsubscribes from a list that is just a list member of a single list. I feel very strongly about that. I don't really care for the reasoning behind why the data is retained. I just think it should be removed for a list member who has no need for an account that manages multiple email address and is subscribed to multiple lists. I host many single lists. So it is very important to me, and as an advocate for my clients, I will state very clearly how important it is to me (and my clients) regards of other user scenarios out there (looking at you Mr. Turnbull). I care about my own. > So perhaps what we should be talking about is UIs for changing user > information, what they would look like and who should be able to change > what. That is a start and I thought I brought that up. We also need a separate conversation on the retention of data apparently. > Note that I personally am a member of many lists, an admin of multiple > lists and a site admin for multiple mailman installations. I am well > aware of the frustrations of list admins who wind up just doing it > because it's way easier than instruction some users as to how to do it > themselves. However, I don't think that is necessarily sufficient reason > to hand over control of global, non-list specific user information to > the admin of one particular list that the user happens to be a member of. I never asked for global control for list owners. You have made that almost a necessity with the multiple email address per user account feature that you brought in. I don't think List owners should have global control but server owners certainly do. But the rightful avoidance of such control for List owners, I think has resulted in a wrongful limiting of what they can do currently. I so disagree with S. Turnbull's disparaging comments that I think we ought to be designing for List owners primarily and not list members/users when it comes to user interfaces. From what I see, it is mostly server and list owners that are interacting with this (mailman-user) list and not list members/users. In my experience, I never hear from list members. Just list owners. Whatever issue list owners have with their own list members are easily handled by them when it comes to Mailman 2. Not so much with Mailman 3. > > Even in mailman 2.1, while a list admin could go to a user's options > page for the list and change things, the "change globally" check boxes > only worked for the user, not for the list admin. > -- Brian Carpenter Harmonylists.com Emwd.com

5 years, 1 month

[MM3-users] Re: signup / registration error - permissions and cert chains

by David Newman

On Dec 31, 2021, at 03:43, Victoriano Giralt <victoriano(a)uma.es> wrote: > > El viernes, 31 de diciembre de 2021 1:48:38 (CET) David Newman escribió: >> I'd like for regular (non-admin) list subscribers to be able to manage >> their subscription preferences and view list archives. > > That's a good way to go :-) > > My response is more of a (very) old sysadmin and Django user (since 2008) > hunch that a proper one based on code and documentation review, but I've been > trying to contribute several times and always (super) Mark Sapiro beats me :-) > >> If I'm reading the error correctly, this is related to an inability to >> verify the cert chain. The /etc/mailman3/settings.py file points to the >> same cert and key files used by Nginx, Postfix, and Dovecot. > > You are right in your diagnose but not in your interpretation (see my comment > below inside the traceback). It is certificate related, but not for server > TLS, but for CLIENT authentication. > > >> EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' >> EMAIL_HOST = 'localhost' >> EMAIL_PORT = 25 >> EMAIL_HOST_USER = 'dnewman(a)networktest.com' >> EMAIL_HOST_PASSWORD = 'wouldnt-you-like-to-know' >> EMAIL_USE_TLS = 'True' >> EMAIL_SSL_CERTFILE = '/etc/ssl/certs/myhost.crt' >> EMAIL_SSL_KEYFILE = '/etc/ssl/private/myhost.key' > > All these settings above are used for SENDING messages and, if I'm not > mistaken, the SSL key and cert are used for authenticating the user sending > the email. Actually, using TLS and SMTP Auth for localhost is a bit too much. > I've been configuring SMTP servers since 1990 and my mail servers just accept > mail form localhost, if they are broken into, the user and password have > already been exposed :-) > >> But this might only be for email, not Postorius/Django. > > You are right (if I also am) > >> What additional configuration is needed to allow regular users to create >> and manage their own accounts? > > I'd say that is more what is not needed (the SMTP TLS authentication) > > I'll remove the "noise". These are the tell tale lines: > >> "/opt/mailman/venv/lib/python3.9/site-packages/django/core/mail/backends/smt >> p.py", line 67, in open >> self.connection.starttls(keyfile=self.ssl_keyfile, >> certfile=self.ssl_certfile) > > The SMTP Django backend is trying to connect to the mail server to send the > Mailman account confirmation message and failing, probably because the user > Django runs as cannot open the private key (which is a very sensible thing if > that private key is the one used for the web facing TLS certificate, I can > tell you how bad in private or search for my name, wasd, apache and VMS ;-)) > > That certificate is not needed for sending email from Django, and, as I said, > not even SMTP Auth for sending via localhost. Actually, doing SMTP Auth on > port 25 is not even recommended practice. Hi Victoriano, Thanks for this. I could use some clarification on what specific changes you are suggesting. I *think* you are saying to remove the EMAIL_USE_TLS stuff and also move to another port (maybe 587), but I am not sure. Also, the reason I added the TLS in the first place was that I was getting errors without it. And I am unclear why the cert / private key pair do not work for Django when they do work OK for Postfix, Nginx, and Dovecot. Thanks for clarifying — and happy and safe 2022 to you as well! dn > > Happy, healthy, safe and well ventilated New Year to all. > > -- > Victoriano Giralt Innovation Director > Digital Transformation Vicerectorate University of Malaga > +34952131415 SPAIN > ================================================================== > Note: signature.asc is the electronic signature of present message > A: Yes. >> Q: Are you sure ? >>> A: Because it reverses the logical flow of conversation. >>>> Q: Why is top posting annoying in email ? > > > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

3 years, 11 months

[MM3-users] Re: Customise Postorius templates?

by Abhilash Raj

> On Feb 7, 2022, at 4:35 AM, Duane Raymond <duane(a)fairsay.com> wrote: > > On Mon, 7 Feb 2022 at 00:56, Abhilash Raj <maxking(a)asynchronous.in> wrote: > >>> On Feb 1, 2022, at 11:01 AM, Mark Sapiro <mark(a)msapiro.net> wrote: >>> >>> On 2/1/22 09:47, Duane Raymond wrote: >>>> Hi, >>>> I'm looking to do some radical customisation of the Postorius (and >> Hyperkitty) templates over the next 6 ish months and was wondering if there >> is a 'best practice' way to do this that will survive updates to MM3. I've >> searched around and not found much and also tested copying >>>> >> venv/lib/python3.7/site-packages/postorius/templates/postorius/lists/summary.html >>>> to >>>> var/templates/lists/summary.html >>>> As suggested in some posts - but it didn't seem to pick up the >> customisation - only customisations in the venv path worked - and they get >> overwritten on update. > > >> The best way to do this would be utilize Django’s template loader. See >> here[1] for the documentation on how you can do this. You want to put the >> path where you are putting the templates under `DIRS` option of the >> `TEMPLATES` section. >> >> Do make sure that you are keeping the directory structure correct so that >> Django can discover them. Like, Hyperkitty’s base.html should be in >> `/custom/path/hyperkitty/base.html`, where `/custom/path` is what you’ve >> setup in DIRS setting above. >> > I hadn’t tried it when I sent email, but when I tried it locally on Postorius, it seems to work for me. > This looks promising! I've not got it working yet, but this is what I've > tried: > > 1. Created folder 'custom' in the mm root (in my case: /opt/mailman/mm/) > 2. Copied the files and folders from postorius/templates/postorius/* and > hyperkitty/templates/hyperkitty/* to the custom/ folder (not the > full postorius/templates/postorius/ match, just evening in the > second postorius/ dirrectory) Note that you want to copy from “src/postorius/templates” directory to your "/opt/mailman/mm/custom” directory. It is important that you retain the “postorius” and “hyperkitty” directories. Here is what I have in my settings.py: - 'DIRS': [], + 'DIRS': ['/Users/maxking/Documents/mm3/postorius/example_project/templates’], And here is the structure of the templates directory you are seeing above: $ tree /Users/maxking/Documents/mm3/postorius/example_project/templates /Users/maxking/Documents/mm3/postorius/example_project/templates └── postorius └── lists └── summary.html Note that I am only overriding the Summary page here. > 3. Recursive update of permissions of the custom/ folder and files to be > mailman > 4. Updated /opt/mailman/mm/settings.py by changing "'DIRS': []" > to "'DIRS': [BASE_DIR, '/custom/']" in the "TEMPLATES =" section Note that you don’t want [“BASE_DIR”, “/custom/“] here, the comma is the wrong value here. The docs I mentioned have a “/“. Although, it is possible that even “/“ since that needs BASE_DIR to be a Path datatype and not string. Easiest thing with least surprise would be to use the full path, atleast to ensure that template discovery works as expected. > 5. Edited the original AND copied postorius/lists/summary.html to have a > small change in the footer of each so I know what version it is serving Since I was running a dev server, it doesn’t need restart, but simply restarting gunicorn should should do it. You don’t need to restart anything else. > 6. Restarted: gunicorn qcluster mailman nginx and cleared browser cache > 7. Reloaded the 'info' page of a few lists to see which version ti was > serving > > Variants: I've also changed the DIRS value to the hard-coded path: > /opt/mailman/mm/custom > > Result: it is still serving the default/original Postorius template vs the > custom one. > > Any suggestions for things I should do to make it use the /custom/ folder > instead? Any recompiling or other actions beyond just restarting? (I > probably don't need to restart qcluster and maybe not even mailman core for > this to take effect but I do it just in case!) Hope that helps. -- thanks, Abhilash Raj (maxking)

3 years, 10 months

[MM3-users] Re: Does hyperkitty show JPEGs in line and what about attachments?

by tlhackque

On 12-Jul-17 12:39, Mark Sapiro wrote: > > Also, with respect to Mailman 2.1, if you want the scrubber to preserve > file names and extensions, set the following in mm_cfg.py > > SCRUBBER_DONT_USE_ATTACHMENT_FILENAME = False > SCRUBBER_USE_ATTACHMENT_FILENAME_EXTENSION = True > > and if you want scrubbed HTML to not be HTML escaped so it renders > rather than looking like raw html, set > > ARCHIVE_HTML_SANITIZER = 3 > > but note this comment from Defaults.py > >> # 3 - Remove text/html as attachments but don't HTML-escape them. Note: this >> # is very dangerous because it essentially means anybody can send an HTML >> # email to your site containing evil JavaScript or web bugs, or other >> # nasty things, and folks viewing your archives will be susceptible. You >> # should only consider this option if you do heavy moderation of your list >> # postings. > > This is an issue with HyperKitty as it appears this is what HyperKitty > does and there's no way to turn it off. > This warning seems a bit dated, though it's not completely wrong. It comes from the days when HTML was new, browsers were fragile, and javascript treated with suspicion. And virus/spam scanners for email were non-existent. Today, it's a very rare website that doesn't rely on javascript (Postorious and Hyperkitty use JS). Browsers, while they still have bugs, are much more defensive. And there are plenty of truly evil sites that they have to defend against. It is certainly true that archived e-mail can turn your site into an unknowing distributor of malware: FLASH bugs, documents with embedded buffer overflows, cross-site scripting and the other many ills of the day. Wikis deal with this frequently. However, in these cases, your mailing list has distributed the same bits to your subscribers - a community that you probably care more about than a random visitor to your (open) archive. I wouldn't run a list - public or private - where the traffic doesn't go through SPAM and virus filtering before Mailman sees it. (SpamAssassin and ClamAV are good open-source solutions.) And once you've done that (and Mailman 3's optional DMARC), most of these attacks are defanged/mitigated. This is essentially automated moderation - to a point. Note that all the Djano authentication schemes packaged with Mailman (facebook, google, etc) rely on javascript and are sites littered with what the comment refers to as "webbugs" - Google Analytics, tracking cookies, browser fingerprinting, 0 size images (the original webbug). They make money (and have become mainstream) using technologies that were considered anti-social when that warning was written. (Personally, I still think of them as anti-social, but the public has chosen to pay for services with privacy...) While some may elect to stick with the highly restrictive policies of "plain text only", this limits the information content and applicability of the the platform. Whether this is acceptable depends on the community that you serve. Mailman can be an effective mechanism to deliver rich media on a "push" basis. And that's "rich" by 1980 standards (bold, well-formatted tables, an attached agenda or document package); not even "rich" by today's (sleeping cat videos...). I think that Mailman has to be able to handle today's rich media with a reasonable degree of safety and convenience. Including in the archives. I thought that was one of the goals for Mailman Version 3... I also think that the advice quoted above should be modified to better reflect these realities. Mailman isn't the only tool available to protect users from evil content, and aggressively filtering to plaintext is a very blunt instrument. Including anti-spam, anti-virus, DNS blacklisting, DKIM/DMARC tests in the delivery pipeline (most of which can be/is done before Mailman touches a post) should be strongly recommended. Checks for headers indicating checked-by local (anti-spam/anti-virus) agents should be available in the Mailman rulesets (and require some cooperation from the MTA to ensure that they can't be passed through from outside.) There is nothing wrong with running a plain text only site, if it serves your community. But if Mailman wants to be relevant in today's environment, it has to adapt to rich content as more than an unwelcome guest. (As I have :0)

8 years, 5 months

[MM3-users] Re: Confirmation emails to Users has wrong domain name (example.com!)

by Odhiambo Washington

On Thu, Sep 30, 2021 at 4:09 PM Abhilash Raj <maxking(a)asynchronous.in> wrote: > > > On Thu, Sep 30, 2021, at 2:20 AM, Odhiambo Washington wrote: > > On Thu, Sep 30, 2021 at 1:59 AM Abhilash Raj <maxking(a)asynchronous.in> > > wrote: > > > >> > >> > >> > On Sep 29, 2021, at 2:34 PM, Odhiambo Washington <odhiambo(a)gmail.com> > >> wrote: > >> > > >> > On Wed, Sep 29, 2021 at 8:31 PM Mark Sapiro <mark(a)msapiro.net> wrote: > >> > > >> >> On 9/29/21 9:50 AM, Odhiambo Washington wrote: > >> >>> 1. Confirmation emails to Users has wrong domain name (example.com > !) > >> >>> <https://docs.mailman3.org/en/latest/faq.html#id1> > >> >>> < > >> >> > >> > https://docs.mailman3.org/en/latest/faq.html#confirmation-emails-to-users-h… > >> >>> > >> >>> > >> >>> This happens when your reverse (SSL) proxy isn’t setting up the > correct > >> >>> headers when proxying requests. Fix this by setting the right > >> >>> proxy_set_header directives: > >> >> ... > >> >>> How is this supposed to be mitigated in Apache when using WSGI? > >> >>> > >> >>> My config: > >> >>> > >> >>> WSGIDaemonProcess hyperkitty threads=25 > python-path=/usr/local/mailman > >> >>> user=mailman group=mailman > >> >>> WSGIPythonHome "/usr/local" > >> >>> WSGIProcessGroup hyperkitty > >> >> > >> >> > >> >> You are using mod_wsgi and not proxying at all, so this is not > relevant > >> >> in your case. > >> >> > >> >> Are you actually seeing this issue? If so, it might be related to > >> >> > >> >> > >> > https://docs.mailman3.org/en/latest/faq.html#the-domain-name-displayed-in-h… > >> >> > >> > > >> > My issue is related to this, but the documentation referred to is not > for > >> > the faint-hearted! > >> > I can't make head or tails of it. > >> > >> Click on “Domain” in Postorius from the top bar, which should take you > to > >> the Domains page. > >> > >> For your domain (in the “Mail Host” column), see the corresponding “Web > >> Host” column, it should look something like: > >> > >> lists.mailman3.org (lists.mailman3.org) > >> (Edit) > >> SITE_ID = 1 > >> > > > > Mine looks different, slightly. But there is no example.com at all. > > > > [image: Abhilash.png] > > > > > > > >> > >> on a new-ish version of Postorius. > >> > > > > I have the newest versions of everything, having installed only > yesterday. > > > > > >> > >> If it doesn’t show the right values and instead shows “example.com” for > >> you, click on the “Edit” link, which will take > >> you to a page that will allow you to edit both the values. > >> > > > > It shows the right values, but with "SITE_ID = 2". In my > settings_local.py > > I have SITE_ID = 1. > > I suppose the example.com is the one tied to SITE_ID = 1 and that is > what I > > have in my settings_local.py. > > That is most likely what is happening. > > > Should I edit my settings_local.py? > > Yes, please update it to have SITE_ID = 2 to correspond to the site you > want and restart. I did this, but there is still a problem: The hyperkitty URL still shows example.com. Maybe I need to re-import the database to clear this? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' :-)

4 years, 2 months

[MM3-users] Re: E-mail every minute: "Cron <www-data@sharky5> ..."

by Odhiambo Washington

On Sun, Jul 7, 2024 at 8:43 PM Robert Heller <heller(a)deepsoft.com> wrote: > At Sun, 7 Jul 2024 19:48:02 +0300 Odhiambo Washington <odhiambo(a)gmail.com> > wrote: > > > > > On Sun, Jul 7, 2024 at 7:16Ã¢Â€Â¯PM Robert Heller <heller(a)deepsoft.com> > wrote: > > > > > At Sun, 7 Jul 2024 18:52:05 +0300 Odhiambo Washington < > odhiambo(a)gmail.com> > > > wrote: > > > > > > > > > > > On Sun, Jul 7, 2024 at 4:12ÃƒÂƒÃ‚Â¢ÃƒÂ‚Ã¢Â‚Â¬ÃƒÂ‚Ã‚Â¯PM Robert > Heller <heller(a)deepsoft.com> > > > wrote: > > > > > > > > > What am I missing? I *think* I have mailman3 *mostly* setup, but > there > > > > > are > > > > > still some configuration things that are missing, but I am not sure > > > how to > > > > > fix > > > > > them (the docs are NOT clear). > > > > > > > > > > > > > Which docs are you relying on? > > > > > > https://docs.mailman3.org/en/latest/config-web.html > > > > > > I presume these are the official docs for mailman3 -- maybe they > aren't? > > > > > > > > > > > How about this - > > > https://docs.mailman3.org/en/latest/install/virtualenv.html > > > > ?? > > > > > > I'm not using a virtual environment. I'm using all native Debian 12 > > > packages, > > > installed via apt. The virtual environment docs are actually even worse > > > (even > > > more confusing). > > > > > > Worse? :-) > > Even more confusing. Both sets of docs make various assumptions and don't > really explain things properly. Like everywhere where "settings.py" is > mentioned, it really means "/etc/mailman3/mailman-web.py" > No! It means /etc/mailman3/settings.py - literally! > In any case, the virtual environment docs are hard to relate to a "native" > install and are generally hard to follow, since they seem to jump all over > the > place. When one day you'll be able to internalize what a Python virtual environment is, you'll realize that it's VERY convenient. You will actually embrace it from that point onwards. (Spaghetti docs?) And it is hard to replace the various (and not > always consistent) virtual environment paths and settings files to the > "native" > paths. Actually, if you're this inclined to run everything natively, MM3 is perhaps not for you. Why? Because you'll not easily find help here. We focus on the virtual environment only as the standard.. Why? Because no one is willing to deal with ALL the OS-centric packaging out there. Python virtual environment is universal across all the OSes, I can say. > The "official" docs are just not useful to me, since I am not using a > virtual > environment. If a virtual environment is recomended, what is the point of > the > Debian 12 packages? We cannot answer that here. I guess they are meant for people like you who strive under pain :-) With the Python virtual environment, I can install and manage MM3 in almost any *nix OS. > Are they just not meant to be used? Really? Do you mean that I should use > a separate package management system for Mailman3? That > really sucks. > Yes, they are meant to be used. Noone denies that. However, they are not packaged by the Mailman Developers. Did you read one response from Mark Sapiro where he said, and I quote: ``` If you prefer to use the Debian packages, that's fine, but if using the Debian packages, your primary resource for support, documentation, bug reports, etc. should be Debian. See https://wiki.list.org/x/12812344 ``` So yes, go ahead and use the Debian packages. No one is stopping you. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

1 year, 5 months

[MM3-users] Re: Digests not working correctly

by Joel Lord

Now I'm on one of the lists in digest mode and I can see that it's a mess. Periodic digests are definitely NOT working, so I'll lay that out here. root@host2:/# cat /etc/cron.d/mailman # This goes in /etc/cron.d/mailman # Replace "apache" by your webserver user ("www-data" on Debian systems) and # set the path to the Django project directory 0 23 * * * lists /usr/local/bin/mailman digests --periodic 0 23 * * * lists /usr/local/bin/mailman notify root@host2:/# grep digests /var/log/cron.log Jun 11 23:00:01 host2 CRON[1632765]: (lists) CMD (/usr/local/bin/mailman digests --periodic) Jun 12 23:00:01 host2 CRON[2177286]: (lists) CMD (/usr/local/bin/mailman digests --periodic) root@host2:/home/members/directory# su - lists lists@host2:~$ /usr/local/bin/mailman digests --periodic lists@host2:~$ ls var/lists/<list>/ digest.mmdf In this case I've got /usr/local/bin/mailman as a symlink to the mailman binary inside the venv's bin directory, just for simplicity. That digest.mmdf file is dated June 9th and clearly ought to have been cleared out on any of the nightly runs between then and today but has not. There are no errors anywhere I can find. How can I try and track this down? -Joel On 6/4/2023 10:15 PM, Joel Lord wrote: > The May 4th digest that went out was _also_ size-triggered, so this may > have nothing to do with periodic digests at all, and possibly my > periodic digests aren't working. I'm not on any of my own lists in > digest mode, I'm slowly extracting diagnostic information out of people > who are. Also, since this is a ~2 month cycle, it's really difficult to > get data points to work with. I'll need to remember to go in and look > when this settles down again (new cycle of activity started last night) > to see if there's anything left pending. > > (venv) root@host2:/home/lists/mailman/venv/bin# pip freeze | grep -i hyper > HyperKitty==1.3.7 > > On 6/4/2023 10:05 PM, Mark Sapiro wrote: >> On 6/4/23 18:35, Joel Lord wrote: >>> >>> The periodic digests do seem to be coming out. I also now have >>> confirmation that the one message in this morning's digest that was >>> from May 4th was also included in the last digest back on May 4th, so >>> it seems that the one message was left behind in the digest queue >>> when the periodic digest was sent. >> >> I don't see how that can happen. The process that sends a digest >> renames the var/lists/<list-id>/digest.mmdf mailbox file in which the >> messages are accumulated to >> var/lists/<list-id>/digest.<volume>.<issue>.mmdf, where <volume> and >> <issue> are the volume and issue numbers of that digest, and then >> queues a message in the `digest` queue to tell the digest runner to >> create the digest from the messages in that mbox and send it. Thus, it >> leaves no var/lists/<list-id>/digest.mmdf mailbox file behind and that >> is created anew when the next post arrives. Further, if there is a >> non-empty digest.mmdf file, its messages should be sent no later than >> the next 11 PM `cron digests`. >> >> >>> There was one earlier message to the list back on May 4th, before the >>> one that got duplicated, but I can't tell if that triggered a >>> size-based digest to be sent: the logs aren't clear enough on that >>> detail for me to tell > >> >> OK >> >> >>> Just to inform things: >>> >>> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep mailman >>> django-mailman3==1.3.9 >>> mailman==3.3.8 >>> mailman-hyperkitty==1.2.1 >>> mailman-web==0.0.6 >>> mailmanclient==3.3.5 >>> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep hyper >>> mailman-hyperkitty==1.2.1 >> >> Actually, it's HyperKitty, not hyperkitty, but I assume HyperKitty is >> up to date as are the others. >> >>> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep post >>> postorius==1.3.8 >>> >>> >> > -- Joel Lord

2 years, 6 months

Search results for query "sapiro"