Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: (venv) mailman-web migrate (as explained) fails with "django.core.exceptions.ImproperlyConfigured: allauth.account.middleware.AccountMiddleware must be added to settings.MIDDLEWARE"

by Mark Sapiro

On 10/19/23 12:12, Eggert Ehmke via Mailman-users wrote: > Hello Mark, > > I am also using MySql/MariaDb, I followed your suggestion. This would uninstall a lot of > other packages, is that safe? I did not continue. > > > $ pip uninstall psycopg2_binary > Found existing installation: psycopg2-binary 2.9.6 > Uninstalling psycopg2-binary-2.9.6: > Would remove: > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2/* > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary-2.9.6.dist-info/* > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/ > libcom_err-2abe824b.so.2.1 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/ > libcrypto-2cd0ac56.so.1.1 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/ > libgssapi_krb5-497db0c6.so.2.2 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/libk5crypto- > b1f99d5c.so.3.1 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/libkeyutils- > dfe70bd6.so.1.5 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/libkrb5- > fcafa220.so.3.3 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/libkrb5support- > d0bcff84.so.0.1 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/ > liblber-69811697.so.2.0.200 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/libldap- > e7812872.so.2.0.200 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/ > libpcre-9513aab5.so.1.2.0 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/ > libpq-227f62f1.so.5.15 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/ > libsasl2-883649fd.so.3.0.0 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/ > libselinux-0922c95c.so.1 > /opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary.libs/ > libssl-0331cfe8.so.1.1 > Proceed (Y/n)? n Yes, it is safe. the things listed other than psycopg2/* are just installation information about psycopg2_binary. No other packages are actually involved. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years

[MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady()

by Mark Sapiro

On 10/2/20 2:58 PM, Alex Schuilenburg via Mailman-users wrote: > > I ran both on the original archive, cleanarch made no changes and > check_hk_import just retported the number of messages. That's actually expected with this mbox as there are apparantly no unescaped `From ` lines in the message body and no other serious defects. > As it turns out, the message was imported, but with the dodgy email as > sender which was fixed by the insert/update as per my comment. > > Hope you have some success with the redacted mbox example. As I mention at <https://gitlab.com/mailman/hyperkitty/-/issues/320#note_422998204> I imported the mbox to a test list, and there were noe real issues. What issue there is is the mbox message's actual From: header is ``` From: Firstname Surname at example-UK <user(a)example.com> ``` Part of what's going on is some MM 2.1 archives has email addresses obfuscated by replacing `@` with ` at ` so the import process reverses this and changes the From: to header is ``` From: Firstname Surname@example-UK <user(a)example.com> ``` which now contains two email addresses albeit not separated by a comma and picks the first. The message then gets properly archived with sender `Firstname Surname@example-UK` which is a syntactically valid email address and doesn't cause the ResponseNotReady() exception in sync_mailman so I don't see that issue. The difference is either the fact that my test runs with the HEAD of the GitLab branch which may not have the issue or the difference between the real From: header and the sanitized one is significant. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years

[MM3-users] Re: Hyperkitty list overview error: thread matching query does not exist

by Mark Sapiro

On 5/23/22 11:11, Darren Hart via Mailman-users wrote: > When attempting to visit the hyperkitty archives list overview for one of my two lists: > > Https://lists.mydomain.com/hyperkitty/list/mylist@mydomain.com/ > > The area where the threads should appear shows the loading spinner (but the activity summary bar chart and most active posters post counts do eventually load). > > The following appears in /var/log/mailman3/web/mailman-web.log: > > > ERROR 2022-05-23 17:50:50,201 1694 django.request Internal Server Error: /hyperkitty/list/mylist(a)mydomain.com/posted-to > Traceback (most recent call last): ... > File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 406, in get > raise self.model.DoesNotExist( > hyperkitty.models.thread.Thread.DoesNotExist: Thread matching query does not exist. > > Viewing “All Threads” or Threads by month (any month) work as expected, displaying the appropriate threads. > > The archives for this list worked previously. There was an incident that resulted in DNS not working and the archiver failing for a few days. The mail that came in during that time does show up in the other views. > > The hyper kitty archives for a second list on this Mailman3 instance is working correctly. > > How can I fix this view of the archives? Or how can I get more information on what might be failing? I'm not sure if this will help, but try running your django management command with arguments `runjob recent_threads_cache`. Note that this will be run daily if you are running the Cron Jobs for Mailman Web. See https://docs.mailman3.org/en/latest/install/virtualenv.html?highlight=perio… -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 4 months

[MM3-users] Re: Importing mbox files into archive defect with lines with From

by Mark Sapiro

On 6/21/19 12:57 AM, andrew.bernard(a)gmail.com wrote: > A curious problem, but seems to be well defined. I am importing mbox format files with 25 years worth of archives in 250 files from a Listserv list into hyperkitty. All is going swimmingly, except I get lots of partial emails with sender of None put into the current month, with text that has clearly been truncated. As I look at each one, I see that the line in the imported text is immediately after a line starting with 'From'. Clearly, 'From:' is an RCS-82 header, but 'From ' is not, and yet there is much evidence here that this is messing up the import. > > Is this a known bug? If not, how do I report it? As other responses in this thread have pointed out, it is a well known issue with mbox format in that lines beginning with 'From ' are message separators. There is a (Python 2) script at <https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/view/head:/bin/cle…> that will process a mbox and prefix with '>' all lines beginning with 'From ' that don't look like real 'From ' lines or which aren't immediately followed by a line that looks like a valid header line. It isn't perfect because it won't handle a message that contains in it's body a copy of another message containing a Unix From_ line, but it can helm with most unescaped From_ lines. If you want to use that script, replace lines 55 and 56 import paths from Mailman.i18n import C_ with def C_(s): return s -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

7 years, 3 months

[MM3-users] Re: integrate subscribe form in Website -> CSRF-token

by Mark Sapiro

On 6/21/21 11:58 AM, Jens Günther wrote: > > We wanted to do the same with Mailman3 (Docker, 3.3.3, Postorius 1.3.4): > > <form method="POST" > action="https://$domain/postorius/lists/$short_lisname.$domain/anonymous_subscribe"> > > <input name="email" value="" type="text"><br> > <input name="email-button" value="Anmelden" type="submit"> > </form> > > When I now enter a mailaddress, the quoted error occurs. This is Django protecting against CSRF. You need to GET the for with the (hidden) CSRF token before you can POST it. > Checking another answer directly to me, there seems to be another hint :-) > > > I believe you are attempting to use the MM Core API via http > GET/POST, rather than via Postorius? That you have tried and get the > CSRF error reported? > > > > Typically this is because you need to add a token to the request to > indicate to the MM Core that your request isn't a forgery. Typically > this token is requested from Core and then presented with subsequent > requests. Have you tried this? > > > > Therefor it is needed to send also a token!? Or use another URL, not > postorius in order to subscribe!? How can I submit such a token? To submit the Token, you need to first GET the form which will have the token. Then you can include it in the POST data. This is not a satisfactory solution for your case. As hinted above, you need to use the REST API. See https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/… -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years, 3 months

[MM3-users] Re: New messages not being archived

by Mark Sapiro

On 7/19/21 9:42 AM, Sarah H wrote: > so there should be inside the docroot created by collectstatic a location /docroot/archives(or hyperkitty)/api? or is there another location that this particular page should be served from? at present the hyperkitty directory there is only css, fonts, img and js dirs. > > if I can find where this should be served from in the file system that'll give me some way to figure out what I'm actually looking for URL wise... Hyperkitty is not serving a web page. HyperKitty has an API at base_url/api with a few endpoints and generates responses to various GETs and POSTs to those endpoints. Mailman core makes calls to methods in mailman_hyperkitty/__init__.py which in turn call the HyperKitty APi and return the result. The `list_url(mlist)` method is called to get the URL to the list's archive for the List-Archive: header. It in turn does a GET for base_url/api/mailman/urls with query fragments key=<api_key> and mlist=<the fully qualified list name> to get the URL The `permalink(mlist, msg)` method is called to get the link to where HyperKitty will store the message for the Archived-At: header. It in turn does a GET for base_url/api/mailman/urls with query fragments key=<api_key>, mlist=<the fully qualified list name> and msgid=<the message-id from the message> to get the URL The `archive_message(mlist, msg)` method is called to actually archive the message. It in turn does a PUT for base_url/api/mailman/archive with the api_key, the fully qualified list name and the text of the message. See https://gitlab.com/mailman/mailman-hyperkitty/-/blob/master/mailman_hyperki… for the full detail. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years, 3 months

[MM3-users] Re: update problems in virtualenv

by Mark Sapiro

On 11/4/25 11:47, Christian H. Kuhn via Mailman-users wrote: > Hi all, > > i followed https://docs.mailman3.org/en/latest/install/virtualenv.html > and ran in different problems. Here's the first. > > All mailman-web runjobs report: > > System check identified some issues: > > WARNINGS: > ?: settings.ACCOUNT_AUTHENTICATION_METHOD is deprecated, use: > settings.ACCOUNT_LOGIN_METHODS = {'username', 'email'} > ?: settings.ACCOUNT_EMAIL_REQUIRED is deprecated, use: > settings.ACCOUNT_SIGNUP_FIELDS = ['email*', 'username*', 'password1*', > 'password2*'] This is https://gitlab.com/mailman/mailman-web/-/issues/37 fixed for the next release. There are two ways to fix this. You can upgrade mailman-web in your venv by activating the venv and doing pip install --upgrade git+https://gitlab.com/mailman/mailman-web Or you can add the following to your settings.py after the imports from mailman_web.settings.base and mailman_web.settings.mailman ``` del ACCOUNT_EMAIL_REQUIRED del ACCOUNT_AUTHENTICATION_METHOD ACCOUNT_SIGNUP_FIELDS = ['username*', 'email*', 'password1*', 'password2*'] ACCOUNT_LOGIN_METHODS = {'email', 'username'} ``` > (venv) mailman@bywater:~/mm/var/data$ pip install --upgrade mailman==3.3.10 > Collecting mailman==3.3.10 ... > INFO: pip is looking at multiple versions of mailman to determine which > version is compatible with other requirements. This could take a while. > ERROR: Could not find a version that satisfies the requirement nntplib; > python_version >= "3.13" (from mailman) (from versions: none) > ERROR: No matching distribution found for nntplib; python_version >= "3.13" > > standard-nntplib 3.13.0 is installed. This is an issue with Mailman 3.3.10 and Python >= 3.13 fixed by https://gitlab.com/mailman/mailman/-/merge_requests/1348 Here again if you want to upgrade Mailman core you can do pip install --upgrade git+https://gitlab.com/mailman/mailman to get the latest from gitlab. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

11 months, 2 weeks

[MM3-users] Re: digest summary heavily cluttered with hrml code

by Mark Sapiro

On 10/14/20 8:46 AM, Mohsen Masoudfar wrote: > Hi, > > I am running Mailman 3.1.1 on Ubuntu 18.04. > > The user complains that the digest summary is cluttered with HTML code. What does 'summary' mean in this context? Are you referring to the 'Today's Topics:' list or the messages themselves? Is this perhaps <https://gitlab.com/mailman/mailman/-/issues/473>? > I thought, I may fix it by: > Members > {user} > Member options > Delivery mode set to [Plain Text Digest]. After setting the value and clicking [save changes], I tried to check the value by getting back to Members setting for the user, it shows the value as "Regular", which was the value before I tried to update it. There are actually 3 sets of user preferences. They are in order of priority, list based, address based and global. You can see these if you are logged in to the web UI as the user and go to 'Mailman settings' in the pull down at the top right. There are issues in Postorius if a user is a member of a list with more than one role - see <https://gitlab.com/mailman/mailman/-/issues/659>. > Question 1: How may I be sure that the value is set, if it is set? Examine the settings with `mailman shell` > Question 2: How may set it for all members of the list? Create a `mailman shell` script. > Question 3: Is there a CLI way to do this? I tried "mailman members " command. But it seems not supporting this option. Again, `mailman shell`, but this is low level and requires some knowledge of Mailman internals. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years

[MM3-users] Re: MTA setup

by Mark Sapiro

On 12/23/24 09:01, Arte Chambers via Mailman-users wrote: > Output from mailman log after sending a test message to the list: > > Dec 23 16:52:23 2024 (684048) Traceback (most recent call last): > File > "/opt/mailman/venv/lib/python3.12/site-packages/mailman_hyperkitty/__init__.py", > line 158, in _archive_message > url = self._send_message(mlist, msg) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/opt/mailman/venv/lib/python3.12/site-packages/mailman_hyperkitty/__init__.py", > line 228, in _send_message > raise ValueError(result.text) > ValueError: > <!doctype html> > <html lang="en"> > <head> > <title>Bad Request (400)</title> > </head> > <body> > <h1>Bad Request (400)</h1><p></p> > </body> > </html> mailman_hyperkitty is sending the message to hyperkitty to be archived and receiving the 400 response. Presumably it has previously requested and received the list_url and the permalink to put in the message headers. This can be confirmed by the presence of List-Archive: and Archived-At: headers with values in the messages from the list. So the configuration seems to be correct to allow mailman_hyperkitty to communicate with hyperkitty. It is not clear why, but this code https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/views/mailma… is getting the ValueError. That can come from one of three places in https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/lib/incoming…. The lack of a Message-ID: is not it because Mailman's lmtp runner will add one. Most likely it is the non-ascii sender address from https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/lib/incoming… What do you get from ``` mailman qfile var/archives/hyperkitty/spool/xxx.pck ``` where xxx.pck is one of the (hopefully smaller) files. In particular, what is the From: header in that? -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 9 months

[MM3-users] Re: I'm seeing a lot of database locked errors

by Mark Sapiro

On 9/11/23 5:00 PM, bryan.kartzman(a)yu.edu wrote: > Thanks. That worked. I converted the sqlite3 database to a postgresql database and I configured the database in the DATABASES definition of the Django settings. The database lock errors have disappeared but I am getting a Django warning of an improper datetime field. I assume that is related to the database conversion. Any advice for a possible solution? These are the error messages: > ... > /usr/lib/python3/dist-packages/django/db/models/fields/__init__.py:1424: RuntimeWarning: DateTimeField MailingList.created_at received a naive datetime (2023-06-18 18:51:09.290256) while time zone support is active. > warnings.warn("DateTimeField %s received a naive datetime (%s)" > ... > /usr/lib/python3/dist-packages/django/db/models/fields/__init__.py:1424: RuntimeWarning: DateTimeField OrmQ.lock received a naive datetime (2023-09-11 20:29:35.702222) while time zone support is active. > warnings.warn("DateTimeField %s received a naive datetime (%s)" > ... These fields and perhaps others need to be defined in Postgresql as timestamp with time zone. E.g. ``` Table "public.hyperkitty_mailinglist" Column | Type | Collation | Nullable | Default ------------------------+--------------------------+-----------+----------+---------------------------------------------------- id | integer | | not null | nextval('hyperkitty_mailinglist_id_seq'::regclass) name | character varying(254) | | not null | display_name | character varying(255) | | not null | description | text | | | subject_prefix | character varying(255) | | not null | archive_policy | integer | | not null | created_at | timestamp with time zone | | not null | list_id | character varying(254) | | | archive_rendering_mode | character varying(255) | | | ... Table "public.django_q_ormq" Column | Type | Collation | Nullable | Default ---------+--------------------------+-----------+----------+------------------------------------------- id | integer | | not null | nextval('django_q_ormq_id_seq'::regclass) key | character varying(100) | | not null | payload | text | | not null | lock | timestamp with time zone | | | Indexes: "django_q_ormq_pkey" PRIMARY KEY, btree (id) ``` -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years, 1 month

Search results for query "sapiro"