Search results for "sapiro" - lists.mailman3.org

[MM3-users]Re: mailman 2 vs. mailman 3: what happened to multipart/digest?

by John Hein

Here's an '02 reference in git to the some heritage for multipart/digest ========== commit 63d2568fe58333f8ca0b7582f74b43e775c581cd Author: bwarsaw <> Date: Thu Jul 11 22:14:25 2002 +0000 send_i18n_digests(): Let's generate MIME digests the correct way, by wrapping each message inside the multipart/digest in a MIMEMessage instance. We also no longer need to initialize the mimedigest instance's payload to the empty list. Also, when sending the digests, be sure to include isdigest=1 in the metadata, otherwise SMTPDirect will try to tack on another footer. ========== John wrote at 16:46 -0600 on May 12, 2016: > Mark Sapiro wrote at 12:18 -0700 on May 12, 2016: > > On 5/12/16 9:16 AM, John wrote: > > > Digest delivery in mailman 2 encapsulated the individual message/rfc822 messages in a multipart/digest container. This allowed email clients to show the messages in the particular digest in a threaded view. > > > > > > This format vs. plain text digests was a user option in Mailman 2. > > > > > > > It seems like the various digest options that I have tried [1] in mailman 3.1.0 (as implemented for the fedora-users mailing list) no longer encapsulate messages in multipart/digest. > > > > > > The core still supports the same plain text and MIME format digests and > > the same individual subscriber and per-list default formats as did > > Mailman 2. The issue is none of these things are exposed in Postorius. > > > > If you subscribe to digests on this list, you will receive the > > 'encapsulated' MIME format because the lists mime_is_default_digest > > setting is True, but currently that has to be set through the > > bin/mailman shell tool. > > > > > > > Is this intentional? Is it a bug - feature that was lost in translation from 2 to 3? Is there a delivery option I'm missing that will turn that back on? > > Thanks for the info - and Barry's reply as well. I understand that > MIME digests are supported in Mailman 3 - that's what I'm using (and > thanks, Barry, for the info that Summary Digests is the same right > now). My quibble is that the multipart/digest encapsulation level is > gone. > > Barry Warsaw wrote at 15:16 -0400 on May 12, 2016: > > The MIME format of MIME digests should roughly be: > > > > multipart/mixed > > text/plain (the digest heading) > > text/plain (the TOC) > > message/rfc822 > > message-1 > > message/rfc822 > > message-2 > > ... > > text/plain (footer) > > > But mm 2 had: > > multipart/mixed > text/plain (the digest heading) > text/plain (the TOC) > multipart/digest <--- this is the part that's gone in mm3 > message/rfc822 > message-1 > message/rfc822 > message-2 > ... > text/plain (footer) > > So was that removal intentional, a bug or ?? > > Maybe you're saying there is definitely an unexposed (in Postorius) > knob for that in mm3. > > > > > It is the intent that Postorius will eventually expose all these > > settings. We're not there yet. At present you could possibly do this by > > interacting with mailmanclient, although I'm not certain the settings > > are exposed there. You can do it via bin/mailman shell if you have > > access and know what you're doing (see > > <http://mailman.readthedocs.io/en/release-3.0/src/mailman/runners/docs/diges…>) > > > > If you file an issue at > > <https://gitlab.com/mailman/postorius/issues/new>, it might help > > expedite this. > > Okay. > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

9 years, 1 month

[MM3-users] Re: Wrong info in moderation mail when held due to match of a header rule

by tlhackque

On 17-Oct-20 18:40, Mark Sapiro wrote: > On 10/17/20 1:31 PM, tlhackque via Mailman-users wrote: >> I might also add the rule number or config filename:line, as in >> >> Header "{name}": rule in {file}:{n} matched value "{value}" >> >> This would be helpful when backtracking from a rejected message to the >> cause. >> >> A sample expansion might be: >> >> Header "X-Spam": rule in foo.conf:83 matched value "yes" > > These rules are not in a configuration file. They are in the headermatch > table in the database and the rules for a particular list are exposed on > the Header filters tab in Postorius or at the REST > lists/LISTNAME/header-matches endpoint. > > Perhaps something like > > {name}: header match rule, pattern "{pattern}" matched value "{value}" > > example: > > subject: header match rule, pattern "sex|long|list|of|other|bad|words" > matched value "This is a post about filtering based on words like sex in > the subject." > > but that could be just adding more clutter. > I considered including the regex in each message, but that seemed redundant - and potentially long. Thus the pointer. Perhaps number the rules for each list, and use that index? The ROWID would seem to be a candidate for the number. Then Postorius could simply number the rules in its tab; the REST API can add the ROWID as a rule_ID field in the returned JSON. (The fact that it's a ROWID rather than an index or unique name is an implementation detail.) If many rules are anticipated, it might want an endpoint or option to fetch a rule by ID. Then it would be easy to drill down on a specific event for diagnosis. e.g. shunt_reason: (or whatever) { cause: "header_match" header: "subject" header_value: "This is a post about filtering based on words like sex in the subject." rule_id: 47 /* or pattern_id? */ } displays as subject: header match rule 47, matched value "This is a post about filtering based on words like sex in the subject." and lists/LISTNAME/header-matches/47 returns { rule_id: 47 pattern: "sex|long|list|of|other|bad|words" position: 3 /* Whatever else is in your table... I haven't looked since I postponed using MM3. eg */ comment:"Required by policy 869.202" last_updated_by: "Corporate Prude" last_updated_on: "1-Apr-1762" }, while lists/LISTNAME/header-matches returns an array of those structures, presumably as today. With those capabilities, the UI can be enhanced so that clicking on a shunted message gives the summary reason, and clicking on the summary gives the regex. I expect that the header and value will be sufficient for understanding an issue most of the time. Looking at the regex is less common, and only necessary when it needs to be updated... I still think that it is more readable and less verbose to order the message as a declarative sentence: Header "subject": rule 47 matched value "sex" but that may be a matter of personal taste. The UI can sort that out... Note that from your description, using ROWID may cause rules in a particular list to have gaps in the rule numbering. I don't think that's a big issue, though Postorious could paper that over - perhaps use the position attribute. Doesn't seem worth doing something more complex. The advantage is that it doesn't require any DB change now. But aliasing it to "rule_id" allows for any future demand to name the rules or use some other numbering scheme. I may have some details wrong - I found mm3 not ready for my needs and am thus out of date - but I hope the high level approach that I outlined is useful.

4 years, 8 months

[MM3-users] Re: Mailman stopping message distribution because of SMTP errors

by Victoriano Giralt

El mié, 12-02-2020 a las 13:28 -0800, Mark Sapiro escribió: > On 2/11/20 10:28 PM, Victoriano Giralt wrote: > > > > I have recently migrated some lists from Mailman 2 to Mailman 3. Users > > have > > been added using the mailman client as pre-verified addresses so the > > migration happened "silently". Thus, there was a dozen "bad" addresses > > (domains that did not exist anymore). > > > It seems you rolled your own instead of using `mailman import21`. Is > there a reason for this? Yes, no access to the previous server command line. So, to tell the truth it was not a "real" migration, my excuses. > > Unfortunately, Mailman got some of the "bad" addresses in the first > > message > > batch for sending out, so, Postfix closed the connection because of > > "too > > many errors" and this stopped the process completely, i.e. Mailman did > > not > > continue sending messages to the rest of the list over 600 addresses. > > > Is the message shunted? Is there a traceback and a shunting message in > mailman.log? > > Normally, I would expect the message to be queued in Mailman's retry > queue and retried by Mailman Yes, it was being retried every 15 minutes with the same result: stopped distribution after the first attempt with the first address batch. Also reprocessing the "retry" files with mailman command interface. > > There was no more information in the logs than the Postfix error > > messages > > in smtp.log. My bad, the errors where in the system maillog where Postfix complains. > What was in the smtp.log? This (redacted to protect the innocent): Feb 05 16:29:44 2020 (86791) ('127.0.0.1', 51146) recip: listname @example.org Feb 05 16:30:16 2020 (86797) <messageid(a)example.com> smtp to listname @example.org for 634 recips, completed in 29.087315797805786 seconds Feb 05 16:30:16 2020 (86797) <messageid(a)example.com> post to listname @example.org from sender(a)example.com, 4354 bytes, 9 failures Feb 05 16:30:20 2020 (86791) ('127.0.0.1', 51642) Data: b'RCPT TO:<listname -bounces(a)example.org>' Feb 05 16:30:20 2020 (86791) ('127.0.0.1', 51642) recip: listname -bounces(a)example.org Feb 05 16:30:43 2020 (86791) ('127.0.0.1', 51870) Data: b'RCPT TO:<listname -bounces(a)example.org>' Feb 05 16:30:43 2020 (86791) ('127.0.0.1', 51870) recip: listname -bounces(a)example.org ... rinse and repeat every 15 minutes ... > Nothing in mailman.log? Nothing at all about SMTP/LMTP > Nothing in var/queue/shunt? Nothing there, a .psv file in var/spool/mailman/bad/ and two .pck files in retry/ > Once the "bad" addresses were removed, the list started working as > > expected. > > > > I consider this a bug. The bouncing addresses should have been marked > > as > > "non deliverable" and the subscriptions "paused" and the rest should > > have > > been delivered. Or, the list owner should have been notified that there > > was > > a problem with message distribution. None of these happened. > > > The HEAD of the https://gitlab.com/mailman/mailman master branch > implements bounce processing. This will be in 3.3.1 which is not yet > released. > > The place for a bug report is <https://gitlab.com/mailman/mailman/issues> Well, if 3.3.1 has bounce processing, I'd rather wait and test when it is released instead of adding "noise" in the bug tracker, right? My problem was solved with a workaround: find and remove the "bad" addresses. Let's leave this as reference in the archives in case someone hits the same problem. -- Victoriano Giralt CIO University of Malaga +34952131415 SPAIN ================================================================== Note: signature.asc is the electronic signature of present message A: Yes. > Q: Are you sure ? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting annoying in email ?

5 years, 4 months

[MM3-users] Re: Mailman, etc. upgrade woes and persistent bugs

by Abhilash Raj

On Sun, Feb 14, 2021, at 8:59 AM, Allan Hansen wrote: > Mark, > > Thank you for this reminder, but it’s not really sufficient for most. > The ‘Mailman Settings’ dialog does, indeed show the other preferences > tags, and yes, as you say, the dialog for ‘List-based preferences’ only > shows values that are not inherited and when you set a value there, it > shows. > > There a two problems with this: > > a. Once a value is overridden on the ‘List-based preferences’ dialog, > you cannot go back to it being inherited by removing the override. Yes! This is something we've known for a while and haven't been able to fix properly. https://gitlab.com/mailman/postorius/-/issues/414 https://gitlab.com/mailman/postorius/-/issues/195 There more than one open issue for this. And one of the reasons it hasn't been fixed yet is that it is larger problem than most low hanging fruits that gets fixed in the available time window for the us to work on issue. This is a long weekend for me, so I'll start some work on this, but not sure if that would be enough to finish. > b. You cannot see, on this dialog, what the inherited value is. https://gitlab.com/mailman/postorius/-/issues/476 I opened an issue for this too, but this might be a little bit longer of a task because the whole inheritance and calculation of effective preference/setting happens in Core. It would require that Core expose the effective values. We do have some calculation of preferences in Postorius to display the "Global Mailman preferences" page for a User. > This is not user-friendly. > > May I suggest the following: > > That each option shows the following: > > o Value 1 > o Value 2 > (or the equivalent for pull-down menus) > > A box showing inheritance status: > [ ] Use inherited value from address-based preference: Value2 > > If the ‘Use inherited value’ is chosen, then the inherited value shows. > but the value radio buttons become non-selectable. It is a good idea, might need Javascript skills that I don't have ;-) I'll still try to take a stab at implementing such a view. > If not, then whatever value the value radio box has become the value > for the list, > > From what I see, the ‘Use inherited value’ is the default for new > lists/addresses. To change a value, the user would > uncheck this box, making the radio buttons selectable, and then set the > desired value. > > Yours, > > Allan > > > > On Feb 12, 2021, at 11:52 , Mark Sapiro <mark(a)msapiro.net> wrote: > > > >> I then looked at several of the disabled accounts, but the subscription info page in Postorius had no information about whether an account was enabled or disabled. Why is this not displayed???. The members can't see if their account is enabled. Is this another example of the disconnect between Mailman and Postorius? > > > > > > If the user has a Django account, she can see all that info at (e.g. for > > this list) <https://lists.mailman3.org/mailman3/accounts/subscriptions/ <https://lists.mailman3.org/mailman3/accounts/subscriptions/>> > > She gets there from `Mailman settings` in the dropdown under her user > > name. She can also get there via the `Manage Subscription` button on the > > list's Info page. That takes her to `List-based preferences` for the > > list. Any setting not selected there is inherited from the Address-based > > preferences or Global Mailman preferences > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

4 years, 4 months

[MM3-users] Re: Mailman3 without Hyperkitty in docker

by Abhilash Raj

On Sat, Mar 2, 2019, at 1:09 AM, Danil Smirnov wrote: > Thank you very much Abhilash! > > Could you set up 'latest' tag in postotius repo <https://hub.docker.com/r/maxking/postorius/tags> please - as it exists for core/web? > I'd like to have consistent approach across the images... Done, triggered a build on Dockerhub. docker.io/maxking/postorius:latest This should be available when the build is done. > > Also it would be really great to extend image labelling used for *quay.io/maxking/mailman-*:rolling* images to Docker Hub ones... Version identification is quite hard for Docker... rolling tags should be available on DockerHub too. They are replicated on both registries. > > With my best regards, > Danil > > сб, 2 мар. 2019 г. в 03:19, Abhilash Raj <maxking(a)asynchronous.in>: >> __ >> >> >> On Thu, Feb 28, 2019, at 10:18 PM, Danil Smirnov wrote: >>> Hi Abhilash, >>> >>> Thank you very much - the previous error is fixed now. >>> >>> But I have another one now, which causes the same 500 Server Error: >>> >>> bash-4.3# cat /opt/mailman-web-data/logs/uwsgi-error.log >>> *** Starting uWSGI 2.0.18 (64bit) on [Fri Mar 1 05:42:09 2019] *** >>> compiled with version: 6.3.0 on 01 March 2019 01:52:42 >>> os: Linux-3.10.0-693.11.1.el7.x86_64 #1 SMP Mon Dec 4 23:52:40 UTC 2017 >>> nodename: mailman-web >>> machine: x86_64 >>> clock source: unix >>> detected number of CPU cores: 2 >>> current working directory: /opt/mailman-web >>> detected binary path: /usr/local/bin/uwsgi >>> !!! no internal routing support, rebuild with pcre support !!! >>> setgid() to 101 >>> setuid() to 100 >>> chdir() to /opt/mailman-web >>> your memory page size is 4096 bytes >>> detected max file descriptor number: 1048576 >>> building mime-types dictionary from file /etc/mime.types...1168 entry found >>> lock engine: pthread robust mutexes >>> thunder lock: disabled (you can enable it with --thunder-lock) >>> uwsgi socket 0 bound to TCP address 0.0.0.0:8080 fd 8 >>> uwsgi socket 1 bound to TCP address 0.0.0.0:8000 fd 9 >>> Python version: 3.6.8 (default, Jan 30 2019, 23:58:16) [GCC 6.3.0] >>> Python main interpreter initialized at 0x55a2df7f57e0 >>> python threads support enabled >>> your server socket listen backlog is limited to 100 connections >>> your mercy for graceful operations on workers is 60 seconds >>> mapped 166752 bytes (162 KB) for 2 cores >>> *** Operational MODE: threaded *** >>> Traceback (most recent call last): >>> File "/usr/local/lib/python3.6/site-packages/django/utils/module_loading.py", line 20, in import_string >>> return getattr(module, class_name) >>> AttributeError: module 'django.contrib.auth.middleware' has no attribute 'SessionAuthenticationMiddleware' >>> >>> The above exception was the direct cause of the following exception: >>> >>> Traceback (most recent call last): >>> File "wsgi.py", line 38, in <module> >>> application = get_wsgi_application() >>> File "/usr/local/lib/python3.6/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application >>> return WSGIHandler() >>> File "/usr/local/lib/python3.6/site-packages/django/core/handlers/wsgi.py", line 136, in __init__ >>> self.load_middleware() >>> File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 34, in load_middleware >>> middleware = import_string(middleware_path) >>> File "/usr/local/lib/python3.6/site-packages/django/utils/module_loading.py", line 24, in import_string >>> ) from err >>> ImportError: Module "django.contrib.auth.middleware" does not define a "SessionAuthenticationMiddleware" attribute/class >>> unable to load app 0 (mountpoint='') (callable not found or import error) >>> *** no app loaded. going in full dynamic mode *** >>> *** uWSGI is running in multiple interpreter mode *** >>> spawned uWSGI master process (pid: 1) >>> spawned uWSGI worker 1 (pid: 37, cores: 2) >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> --- no python application found, check your startup logs for errors --- >>> >>> As per my investigation, this error possibly caused by Django upgrade: >>> >>> *The SessionAuthenticationMiddleware class is removed. It provided no functionality since session authentication is unconditionally enabled in Django 1.10.* >>> >>> See https://docs.djangoproject.com/en/2.0/releases/2.0/ >>> >>> After I've removed the line >>> https://github.com/maxking/docker-mailman/blob/master/postorius/mailman-web… >>> from settings.py, the container is finally working. >>> >>> I've placed PR in github: >>> https://github.com/maxking/docker-mailman/pull/314 >> >> Thank you for your contributions! >> >> I have merged your pull request and it should now be out (hopefully!). >> >>> >>> Thank you for your help. >>> >>> Danil >>> >>> >>> пт, 1 мар. 2019 г. в 04:33, Abhilash Raj <maxking(a)asynchronous.in>: >>>> >>>> >>>> On Thu, Feb 28, 2019, at 5:19 PM, Abhilash Raj wrote: >>>> > On February 27, 2019 11:53:34 PM PST, Danil Smirnov <danil(a)smirnov.la> wrote: >>>> > >Hi Mark, >>>> > > >>>> > >Actually all this stuff happens inside the container after it's started >>>> > >and >>>> > >initialized without errors. I didn't amend the image anyhow. >>>> > > >>>> > >Also I have the configuration based on docker-compose.yaml provided in >>>> > >the >>>> > >repository which works just fine for mailman-web image. >>>> > > >>>> > >The very same config with postorius image (+ UWSGI_STATIC_MAP param) >>>> > >fails >>>> > >with 500 error... >>>> > > >>>> > >Now answering your questions: >>>> > > >>>> > >ср, 27 февр. 2019 г., 22:56 Mark Sapiro <mark(a)msapiro.net>: >>>> > > >>>> > >> do you have django-q in INSTALLED_APPS in your Django settings and >>>> > >what >>>> > >> is set for Q_CLUSTER? >>>> > >> >>>> > > >>>> > >I've checked settings.py inside the container and I haven't found any >>>> > >of >>>> > >the mentioned configurations there. >>>> > > >>>> > >Does the manage.py that's being run here point to the right >>>> > >settings.py? >>>> > >> >>>> > > >>>> > >No clue. >>>> > > >>>> > >Also I wonder why we have some Hyperkitty-related cronjobs in non-HK >>>> > >configured image? >>>> > > >>>> > >https://github.com/maxking/docker-mailman/blob/master/postorius/mailman-web… >>>> > >>>> > Because I am a dum dum ;-) >>>> > >>>> > Will work on removing those crons and stuff. Would be nice if you >>>> > opened an issue on Github so that I don't forget. >>>> >>>> Should now be fixed in the latest image. >>>> >>>> > >>>> > >Danil >>>> > > >>>> > > >>>> > > >>>> > >> -- >>>> > >> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, >>>> > >> San Francisco Bay Area, California better use your sense - B. >>>> > >Dylan >>>> > >> _______________________________________________ >>>> > >> Mailman-users mailing list -- mailman-users(a)mailman3.org >>>> > >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org >>>> > >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >>>> > >> >>>> > >_______________________________________________ >>>> > >Mailman-users mailing list -- mailman-users(a)mailman3.org >>>> > >To unsubscribe send an email to mailman-users-leave(a)mailman3.org >>>> > >https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >>>> > >>>> > >>>> > -- >>>> > Sent from my Android device with K-9 Mail. Please excuse my brevity. >>>> >>>> -- >>>> thanks, >>>> Abhilash Raj (maxking) >> >> -- >> thanks, >> Abhilash Raj (maxking) >> >> -- thanks, Abhilash Raj (maxking)

6 years, 4 months

[MM3-users] Re: Timeouts

by tlhackque

On 25-Apr-21 20:34, Mark Sapiro wrote: > On 4/25/21 4:37 PM, tlhackque via Mailman-users wrote: > >> The described timeouts are something that hyperkitty ought to be able to >> avoid. For apache, the timeout is idle time between blocks of output. >> Hyperkitty can avoid this by generating the archive in segments (based >> on size, or elapsed time), flushing its output buffer, generating a >> multi-file archive, and/or using Transfer-Encoding: chunked (chunked >> doesn't work for http/2). It ought to be able to break the work into >> blocks of "n" messages & do something to generate output. Besides >> avoiding timeouts, working in segments allows the GUI to display >> meaningful progress (e.g. if you're loading with XMLHttpRequest, >> "onprogress") It really oughtn't be up to the user to break up the >> request. > It is not the web server that times out. I'm not sure about uwsgi > because I don't use it, but the timeouts I see are on servers that use > gunicorn as the WSGI interface to Django and the timeout is in a > gunicorn worker. This is controlled by the timeout setting in the > gunicorn config. <https://docs.gunicorn.org/en/stable/settings.html#timeout> > > Note that even 300 seconds is not enough to download the entire > <https://mail.python.org/archives/list/python-dev@python.org/> archive. > > It may be possible to get HyperKitty to chunk the output to avoid this, > but it doesn't currently do that. Care to submit an MR? > > I'm afraid (u)WSGI, Django, and gunicorn are not technologies that I work with. It sounds as if hyperkitty is compiling the entire archive before sending the first byte. The gunicorn doc that you pointed to says Workers silent for more than this many seconds are killed and restarted. Setting it to 0 has the effect of infinite timeouts by disabling timeouts for all workers entirely. "Silent" sounds like the standard webserver "you have to push some bits, or we assume you're stuck". My understanding is that gunicorn is a Python persistence server that is run behind a webserver proxy. So the (proxy) webserver (apache, nginx, ...) timeouts also apply and would need to be increased. Might be interesting to try 0 (gunicorn) / 1200 (webserver) with your python-dev archive, time it and see how much (encoded) data is transferred... (I would expect most mailing list archives to compress nicely, though those with binary attachments wont.) But fiddling with timeouts is treating the symptom, not the root cause. The right solution is to stream, segment (or chunk) the output, because in the general case, no timeout is long enough. It'll always be possible to find an archive that's just one byte (or second) longer than any chosen timeout. (See the halting problem.) You want the timeout to catch a lack of progress, not total time that's a function of transaction size. (Webservers may also have limits on transaction size - e.g. mod_security, but they're only useful when the upper bound on a response is knowable.) Thus, the timeout(s) should be roughly independent of the archive size; on the order of time-to-first-byte (which ordinarily is longer than time between segments/chunks). Also note that streaming requires fewer server resources than compiling a complete archive before sending, since you don't need to create the entire archive in memory (or in a tempfile). You only need enough memory to efficiently buffer the file I/O and to contain the compression tables/output buffer. Except for trivial cases, this will be independent of the archive size. The only downside is that if the comm link is slow, you may hold a reader lock on the source data for longer than necessary with the current scheme. Seems as though this deserves an issue. I guess I could open one - but you have the experience/test cases.

4 years, 2 months

[MM3-users] Re: Problem with missing MySQLDatabase

by Peter Sørensen

Hi Mark, Unfortunately I answered on ³auto response not sent² instead of this - sorry. In the meantime I found one problem. The comment I¹ve put at the end of line ## MOD .. This is not the way to do it in Python. I normally use Perl and not a Python as script language. Deleteing the #Š Made this problem disappear but instead below problem showed up. /etc/mailman3# mailman aliases Traceback (most recent call last): File /usr/bin/mailman, line 11, in <module> load_entry_point('mailman==3.1.1', 'console_scripts', 'mailman')() File /usr/lib/python3/dist-packages/mailman/bin/mailman.py, line 94, in main initialize(config_path) File /usr/lib/python3/dist-packages/mailman/core/initialize.py, line 189, in initialize initialize_2(propagate_logs=propagate_logs) File /usr/lib/python3/dist-packages/mailman/core/initialize.py, line 160, in initialize_2 config.db = getUtility(IDatabaseFactory, utility_name).create() File /usr/lib/python3/dist-packages/mailman/database/factory.py, line 54, in create database.initialize() File /usr/lib/python3/dist-packages/mailman/database/base.py, line 106, in initialize self.engine = create_engine(url, isolation_level='READ UNCOMMITTED') File /usr/lib/python3/dist-packages/sqlalchemy/engine/__init__.py, line 387, in create_engine return strategy.create(*args, **kwargs) File /usr/lib/python3/dist-packages/sqlalchemy/engine/strategies.py, line 80, in create dbapi = dialect_cls.dbapi(**dbapi_args) File /usr/lib/python3/dist-packages/sqlalchemy/dialects/mysql/pymysql.py, line 62, in dbapi return __import__('pymysql') ModuleNotFoundError: No module named Œpymysql' I have installed pymysql with: pip install pymysql When I do this once more I get pip install pymysql Requirement already satisfied: pymysql in /usr/local/lib/python2.7/dist-packages Requirement already satisfied: cryptography in /usr/lib/python2.7/dist-packages (from pymysql) So the problem has probably to do with this installed in python2.7 and not python3. How do I solve that ? Best regards Peter Sørensen University of Southern Denmark Den 19/10/18 19:13 skrev "Mark Sapiro" <mark(a)msapiro.net>: >On 10/19/18 3:44 AM, Peter Sørensen wrote: >> Hi, >> >> I have looked into installing this before without success but found >> The Debian package Mailman3-full >> >> I installed this succesfully along with the mailman-suite. >> I have configured varios setting but are stuck when trying to generate >>aliases >> For mailman with: >> >> mailman aliases >> Traceback (most recent call last): >> File /usr/bin/mailman, line 11, in <module> >> load_entry_point('mailman==3.1.1', 'console_scripts', 'mailman')() >> File /usr/lib/python3/dist-packages/mailman/bin/mailman.py, line 94, >>in main >> initialize(config_path) >> File /usr/lib/python3/dist-packages/mailman/core/initialize.py, line >>189, in initialize >> initialize_2(propagate_logs=propagate_logs) >> File /usr/lib/python3/dist-packages/mailman/core/initialize.py, line >>160, in initialize_2 >> config.db = getUtility(IDatabaseFactory, utility_name).create() >> File /usr/lib/python3/dist-packages/mailman/database/factory.py, line >>52, in create >> database = call_name(database_class) >> File /usr/lib/python3/dist-packages/mailman/utilities/modules.py, >>line 67, in call_name >> named_callable = find_name(dotted_name) >> File /usr/lib/python3/dist-packages/mailman/utilities/modules.py, >>line 50, in find_name >> __import__(package_path) >> ModuleNotFoundError: No module named >>'mailman.database.mysql.MySQLDatabase\t\t\t#MOD(inserted ) 02'; >>'mailman.database.mysql' is not a package >> >> I can see from the message that I¹m missing a module but I can¹t figure >>out. how to get >> This installed. The mudule is referenced in the mailman.cfg as >> class: mailman.database.mysql.MySQLDatabase > > >The module mailman.database.mysql is part of mailman core. In your case, >it should be /usr/lib/python3/dist-packages/mailman/database/mysql.py. >If that file exists, I don't know what the problem might be. If not, >there is something wrong with your installation. In any case, this is a >Debian issue. You need to take it up with Debian. > >-- >Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, >San Francisco Bay Area, California better use your sense - B. Dylan >_______________________________________________ >Mailman-users mailing list -- mailman-users(a)mailman3.org >To unsubscribe send an email to mailman-users-leave(a)mailman3.org >https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

6 years, 8 months

[MM3-users] Re: Mailman installation results in KeyError: 'env_py'

by Abhilash Raj

On Sun, Oct 22, 2017, at 09:36 AM, Marvin Gülker wrote: > Am 22. October 2017 um 07:49 Uhr -0700 schrieb Mark Sapiro > <mark(a)msapiro.net>: > > or you can install the fix at > > Thanks. Applying this patch makes it work. Now I've got a different > problem, though. I can't get it to connect to Postfix for sending > emails it appears. After I created a superuser account using > > $ python manage.py createsuperuser > > as described on <http://postorius.readthedocs.io/en/latest/setup.html>, > I tried to login using this credentials into Postorius. It then reported > it sent a confirmation email to me (for an account I created on > commandline?!), but Postfix' logs reveal that it didn't; there's not > even a sign of an attempted delivery in /var/log/mail.log. I've > configured the Postfix configuration in mailman.cfg like this: > > [mta] > incoming: mailman.mta.postfix.LMTP > outgoing: mailman.mta.deliver.deliver > lmtp_host: 127.0.0.1 > lmtp_port: 8024 > smtp_host: localhost > smtp_port: 25 > configuration: python:mailman.config.postfix > > as was outlined on > <http://mailman.readthedocs.io/en/latest/src/mailman/docs/mta.html>. > > mailman-suite's settings.py contains: > > # Change this when you have a real email backend > EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' > EMAIL_HOST = 'localhost' > EMAIL_PORT = 25 > > which was described on <http://docs.list.org/en/latest/config-web.html>. > > For completeness, the following is in Postfix' main.cf, but is not the > cause of the problem as it affects the incoming direction, whereas I > refer to the outgoing one: > > transport_maps = hash:/home/mailman/var/data/postfix_lmtp > local_recipient_maps = hash:/home/mailman/var/data/postfix_lmtp > > I failed to locate any kind of logging information about the failed > email, so I don't even know where I should start looking for the > problem. var/logs/smtp.log is empty in the mailman directory, and > likewise is logs/mailmansuite.log in the mailman-suite_project > directory. As Simon mentioned, try using DEBUG=False, if you don't already have that. With DEBUG=True, you should see emails printed in a `emails` directory under `mailman-suite_project` directory. This is a confirmation email to verify the email addresses, which happens even if you create an account from command line. > That being said, the enormous number of configuration files to consider > and dozens of cross-references to other pages in the docs make the > installation guide hard to follow; it also lacks logical structure in my > opinion, when it refers to parts of the configuration which are later > explained. For someone like me who never programmed a Django application > (Ruby dev) this is all pretty cryptic. Especially, that some parts of > information are on <http://docs.list.org/en/latest/index.html> whereas > other parts are scattered over readthedocs.io makes it very > complicated. There are only two primary configuration files (there are a few more for finer configurations ;-): - mailman.cfg: This is the configuration for the Core engine and can be located any of the places mentioned in [1] or pointed to by environment var MAILMAN_CONFIG_FILE. - settings.py: This is the configuration file for Django under `mailman-suite_project` and controls the frontend (Postorius & Hyperkitty). Django is a huge framework with literally 1000s of options. It would be impossible to copy that information from Django's documentation and keep it updated in our docs. Hence, we have links to appropriate places when needed. Same goes out for the rest of the libraries that we use. All the settings mentioned in the documentation go to either of these files, except, when configuring web server and mail server. Please open an issue[2] if it is not clear where a particular configuration snippet goes and we will fix it. [1]: http://docs.mailman3.org/en/latest/config-core.html [2]: https://gitlab.com/mailman/mailman-suite-doc/issues Hope that helps! -- Abhilash Raj maxking(a)asynchronous.in

7 years, 8 months

[MM3-users] mailman-web migrate fails: KeyError: 'django'

by David Newman

Following the venv installation docs on a Debian 10 system, the command "mailman-web migrate" fails with a "KeyError: 'django'" error. Output pasted below [1]. There's an earlier thread on this error [2] but I think the cause may be different. Even so, per Mark Sapiro's suggestion I changed STATIC_ROOT to /opt/mailman/mm/var/static in settings.py. In mailman.cfg, var_dir is at the default setting of /opt/mailman/mm/var/static. I previously had tried creating /opt/mailman/mm/web/logs and chown'ing it to mailman, but "mailman-web migrate" failed with the same error. FWIW, mailman-web and hyperkitty installation initially failed with the suggesting that I upgrade pip, which I did using " pip install --upgrade pip". Thanks in advance for troubleshooting clues. dn [1] (venv) mailman@somehost:~$ mailman-web migrate Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.7/site-packages/django/template/utils.py", line 66, in __getitem__ return self._engines[alias] KeyError: 'django' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.7/site-packages/django/template/backends/django.py", line 121, in get_package_libraries module = import_module(entry[1]) File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/opt/mailman/venv/lib/python3.7/site-packages/hyperkitty/templatetags/decorate.py", line 4, in <module> from hyperkitty.lib.renderer import markdown_renderer, text_renderer File "/opt/mailman/venv/lib/python3.7/site-packages/hyperkitty/lib/renderer.py", line 7, in <module> from mistune.scanner import escape_html, escape_url ImportError: cannot import name 'escape_html' from 'mistune.scanner' (/opt/mailman/venv/lib/python3.7/site-packages/mistune/scanner.py) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/opt/mailman/venv/lib/python3.7/site-packages/mailman_web/manage.py", line 30, in main execute_from_command_line(sys.argv) File "/opt/mailman/venv/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line utility.execute() File "/opt/mailman/venv/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/mailman/venv/lib/python3.7/site-packages/django/core/management/base.py", line 328, in run_from_argv self.execute(*args, **cmd_options) File "/opt/mailman/venv/lib/python3.7/site-packages/django/core/management/base.py", line 366, in execute self.check() File "/opt/mailman/venv/lib/python3.7/site-packages/django/core/management/base.py", line 395, in check include_deployment_checks=include_deployment_checks, File "/opt/mailman/venv/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 64, in _run_checks issues.extend(super()._run_checks(**kwargs)) File "/opt/mailman/venv/lib/python3.7/site-packages/django/core/management/base.py", line 382, in _run_checks return checks.run_checks(**kwargs) File "/opt/mailman/venv/lib/python3.7/site-packages/django/core/checks/registry.py", line 72, in run_checks new_errors = check(app_configs=app_configs) File "/opt/mailman/venv/lib/python3.7/site-packages/django/contrib/admin/checks.py", line 76, in check_dependencies for engine in engines.all(): File "/opt/mailman/venv/lib/python3.7/site-packages/django/template/utils.py", line 90, in all return [self[alias] for alias in self] File "/opt/mailman/venv/lib/python3.7/site-packages/django/template/utils.py", line 90, in <listcomp> return [self[alias] for alias in self] File "/opt/mailman/venv/lib/python3.7/site-packages/django/template/utils.py", line 81, in __getitem__ engine = engine_cls(params) File "/opt/mailman/venv/lib/python3.7/site-packages/django/template/backends/django.py", line 25, in __init__ options['libraries'] = self.get_templatetag_libraries(libraries) File "/opt/mailman/venv/lib/python3.7/site-packages/django/template/backends/django.py", line 43, in get_templatetag_libraries libraries = get_installed_libraries() File "/opt/mailman/venv/lib/python3.7/site-packages/django/template/backends/django.py", line 108, in get_installed_libraries for name in get_package_libraries(pkg): File "/opt/mailman/venv/lib/python3.7/site-packages/django/template/backends/django.py", line 125, in get_package_libraries "trying to load '%s': %s" % (entry[1], e) django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'hyperkitty.templatetags.decorate': cannot import name 'escape_html' from 'mistune.scanner' (/opt/mailman/venv/lib/python3.7/site-packages/mistune/scanner.py) [2] https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…

3 years, 7 months

[MM3-users] Re: Badsignature on uwsgi-error.log

by David Partain

Hi Mark & all, Thanks for the response. Just picking up where we are at the moment to provide a bit more info, in case you have ideas. Thanks, Stephen T. as well for the idea of turning on DEBUG. We might resort to that in the end. This is what our system is running right now: Mailman Core Version GNU Mailman 3.3.9 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] As previously noted, we moved this off of an old Ubuntu package-based installation that has delivered mail reliably but many other things haven't worked well, so we're happy to be on something that might improve that situation. Right now, archiving is not working on the new installation, both building the archive on the new installation after the move and for new messages. uwsgi-error.log is getting blasted with constant errors along the lines of what follows. I suspect that these errors are why archiving isn't working, but I obviously don't know that: Message: BadSignature('Signature "ZWBXqQmWZwHTfa390lLIacpp8NQ" does not match') Arguments: ('Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py", line 356, in pusher task = SignedPackage.loads(task[1]) File "/local/mailman/venv/lib/python3.10/site-packages/django_q/signing.py", line 25, in loads return signing.loads( File "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py", line 35, in loads base64d = force_bytes(TimestampSigner(key, salt=salt).unsign(s, max_age=max_age)) File "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py", line 70, in unsign result = super(TimestampSigner, self).unsign(value) File "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py", line 55, in unsign raise BadSignature(\'Signature "%s" does not match\' % sig) django.core.signing.BadSignature: Signature "ZWBXqQmWZwHTfa390lLIacpp8NQ" does not match',) What we have in our config files on the old installation and new installation are (the quotes are exactly as they are in the files): Old installation: mailman-web.py SECRET_KEY = 'somerandomstringiaintputtinginmail' MAILMAN_ARCHIVER_KEY = 'somerandomstringiaintputtinginmail' mailman-hyperkitty.cfg api_key: somerandomstringiaintputtinginmail New installation: settings.py SECRET_KEY = 'somerandomstringiaintputtinginmail' MAILMAN_ARCHIVER_KEY = 'somerandomstringiaintputtinginmail' mailman-hyperkitty.cfg api_key: somerandomstringiaintputtinginmail All of that says that the value is the same for all three (whether I should have done so or not...) and those settings were carried along to the new installation as well. Django is a big scary beast as I've not had to manage django apps before. Would it make sense that the error above is because what I _think_ django believes is the SECRET_KEY (in the config files above) is in fact not what django believes? Maybe it was set during installation (which I didn't do)? I don't believe anyone has purposefully changed it. I suppose I could do something like https://medium.com/django-unleashed/securing-django-applications-best-pract… to reset the secret key, but I'm not doing that on Friday afternoon :) If anyone has other thoughts, please holler. Wishing you all a good weekend from lighter-by-the-day Sweden. Cheers, David -----Original Message----- From: Mark Sapiro <mark(a)msapiro.net<mailto:Mark%20Sapiro%20%3cmark@msapiro.net%3e>> To: mailman-users(a)mailman3.org<mailto:mailman-users@mailman3.org> Subject: [MM3-users] Re: Badsignature on uwsgi-error.log Date: Wed, 20 Mar 2024 18:12:20 -0700 On 3/20/24 14:25, Helio Loureiro wrote: Hi, I keep receiving a lot of errors like this: --- Logging error --- Traceback (most recent call last): ... "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py", line 55, in unsign raise BadSignature('Signature "%s" does not match' % sig) django.core.signing.BadSignature: Signature "F6V0Dr_SvBsx5-cY8vcvLXrX8tA" does not match ... The api_key on mailman-hypperkitty.cfg, SECRET_KEY and MAILMAN_ARCHIVE_KEY on setting.py are the same. Of those three, the api_key on mailman-hypperkitty.cfg and MAILMAN_ARCHIVE_KEY must match. SECRET_KEY is different. It is a Django thing and has nothing to do with the others. See https://docs.djangoproject.com/en/5.0/ref/settings/#secret-key If you changed it, that's the issue. If you know what it was previously, you could set the previous value in SECRET_KEY_FALLBACKS, see https://docs.djangoproject.com/en/5.0/ref/settings/#secret-key-fallbacks

1 year, 3 months

Search results for query "sapiro"