Re: Mailman3 without Hyperkitty in docker
by Abhilash Raj
On Sat, Mar 2, 2019, at 1:09 AM, Danil Smirnov wrote:
> Thank you very much Abhilash!
>
> Could you set up 'latest' tag in postotius repo <https://hub.docker.com/r/maxking/postorius/tags> please - as it exists for core/web?
> I'd like to have consistent approach across the images...
Done, triggered a build on Dockerhub.
docker.io/maxking/postorius:latest
This should be available when the build is done.
>
> Also it would be really great to extend image labelling used for *quay.io/maxking/mailman-*:rolling* images to Docker Hub ones... Version identification is quite hard for Docker...
rolling tags should be available on DockerHub too. They are replicated on both registries.
>
> With my best regards,
> Danil
>
> сб, 2 мар. 2019 г. в 03:19, Abhilash Raj <maxking(a)asynchronous.in>:
>> __
>>
>>
>> On Thu, Feb 28, 2019, at 10:18 PM, Danil Smirnov wrote:
>>> Hi Abhilash,
>>>
>>> Thank you very much - the previous error is fixed now.
>>>
>>> But I have another one now, which causes the same 500 Server Error:
>>>
>>> bash-4.3# cat /opt/mailman-web-data/logs/uwsgi-error.log
>>> *** Starting uWSGI 2.0.18 (64bit) on [Fri Mar 1 05:42:09 2019] ***
>>> compiled with version: 6.3.0 on 01 March 2019 01:52:42
>>> os: Linux-3.10.0-693.11.1.el7.x86_64 #1 SMP Mon Dec 4 23:52:40 UTC 2017
>>> nodename: mailman-web
>>> machine: x86_64
>>> clock source: unix
>>> detected number of CPU cores: 2
>>> current working directory: /opt/mailman-web
>>> detected binary path: /usr/local/bin/uwsgi
>>> !!! no internal routing support, rebuild with pcre support !!!
>>> setgid() to 101
>>> setuid() to 100
>>> chdir() to /opt/mailman-web
>>> your memory page size is 4096 bytes
>>> detected max file descriptor number: 1048576
>>> building mime-types dictionary from file /etc/mime.types...1168 entry found
>>> lock engine: pthread robust mutexes
>>> thunder lock: disabled (you can enable it with --thunder-lock)
>>> uwsgi socket 0 bound to TCP address 0.0.0.0:8080 fd 8
>>> uwsgi socket 1 bound to TCP address 0.0.0.0:8000 fd 9
>>> Python version: 3.6.8 (default, Jan 30 2019, 23:58:16) [GCC 6.3.0]
>>> Python main interpreter initialized at 0x55a2df7f57e0
>>> python threads support enabled
>>> your server socket listen backlog is limited to 100 connections
>>> your mercy for graceful operations on workers is 60 seconds
>>> mapped 166752 bytes (162 KB) for 2 cores
>>> *** Operational MODE: threaded ***
>>> Traceback (most recent call last):
>>> File "/usr/local/lib/python3.6/site-packages/django/utils/module_loading.py", line 20, in import_string
>>> return getattr(module, class_name)
>>> AttributeError: module 'django.contrib.auth.middleware' has no attribute 'SessionAuthenticationMiddleware'
>>>
>>> The above exception was the direct cause of the following exception:
>>>
>>> Traceback (most recent call last):
>>> File "wsgi.py", line 38, in <module>
>>> application = get_wsgi_application()
>>> File "/usr/local/lib/python3.6/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
>>> return WSGIHandler()
>>> File "/usr/local/lib/python3.6/site-packages/django/core/handlers/wsgi.py", line 136, in __init__
>>> self.load_middleware()
>>> File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 34, in load_middleware
>>> middleware = import_string(middleware_path)
>>> File "/usr/local/lib/python3.6/site-packages/django/utils/module_loading.py", line 24, in import_string
>>> ) from err
>>> ImportError: Module "django.contrib.auth.middleware" does not define a "SessionAuthenticationMiddleware" attribute/class
>>> unable to load app 0 (mountpoint='') (callable not found or import error)
>>> *** no app loaded. going in full dynamic mode ***
>>> *** uWSGI is running in multiple interpreter mode ***
>>> spawned uWSGI master process (pid: 1)
>>> spawned uWSGI worker 1 (pid: 37, cores: 2)
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>> --- no python application found, check your startup logs for errors ---
>>>
>>> As per my investigation, this error possibly caused by Django upgrade:
>>>
>>> *The SessionAuthenticationMiddleware class is removed. It provided no functionality since session authentication is unconditionally enabled in Django 1.10.*
>>>
>>> See https://docs.djangoproject.com/en/2.0/releases/2.0/
>>>
>>> After I've removed the line
>>> https://github.com/maxking/docker-mailman/blob/master/postorius/mailman-web…
>>> from settings.py, the container is finally working.
>>>
>>> I've placed PR in github:
>>> https://github.com/maxking/docker-mailman/pull/314
>>
>> Thank you for your contributions!
>>
>> I have merged your pull request and it should now be out (hopefully!).
>>
>>>
>>> Thank you for your help.
>>>
>>> Danil
>>>
>>>
>>> пт, 1 мар. 2019 г. в 04:33, Abhilash Raj <maxking(a)asynchronous.in>:
>>>>
>>>>
>>>> On Thu, Feb 28, 2019, at 5:19 PM, Abhilash Raj wrote:
>>>> > On February 27, 2019 11:53:34 PM PST, Danil Smirnov <danil(a)smirnov.la> wrote:
>>>> > >Hi Mark,
>>>> > >
>>>> > >Actually all this stuff happens inside the container after it's started
>>>> > >and
>>>> > >initialized without errors. I didn't amend the image anyhow.
>>>> > >
>>>> > >Also I have the configuration based on docker-compose.yaml provided in
>>>> > >the
>>>> > >repository which works just fine for mailman-web image.
>>>> > >
>>>> > >The very same config with postorius image (+ UWSGI_STATIC_MAP param)
>>>> > >fails
>>>> > >with 500 error...
>>>> > >
>>>> > >Now answering your questions:
>>>> > >
>>>> > >ср, 27 февр. 2019 г., 22:56 Mark Sapiro <mark(a)msapiro.net>:
>>>> > >
>>>> > >> do you have django-q in INSTALLED_APPS in your Django settings and
>>>> > >what
>>>> > >> is set for Q_CLUSTER?
>>>> > >>
>>>> > >
>>>> > >I've checked settings.py inside the container and I haven't found any
>>>> > >of
>>>> > >the mentioned configurations there.
>>>> > >
>>>> > >Does the manage.py that's being run here point to the right
>>>> > >settings.py?
>>>> > >>
>>>> > >
>>>> > >No clue.
>>>> > >
>>>> > >Also I wonder why we have some Hyperkitty-related cronjobs in non-HK
>>>> > >configured image?
>>>> > >
>>>> > >https://github.com/maxking/docker-mailman/blob/master/postorius/mailman-web…
>>>> >
>>>> > Because I am a dum dum ;-)
>>>> >
>>>> > Will work on removing those crons and stuff. Would be nice if you
>>>> > opened an issue on Github so that I don't forget.
>>>>
>>>> Should now be fixed in the latest image.
>>>>
>>>> >
>>>> > >Danil
>>>> > >
>>>> > >
>>>> > >
>>>> > >> --
>>>> > >> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
>>>> > >> San Francisco Bay Area, California better use your sense - B.
>>>> > >Dylan
>>>> > >> _______________________________________________
>>>> > >> Mailman-users mailing list -- mailman-users(a)mailman3.org
>>>> > >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>>>> > >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>>> > >>
>>>> > >_______________________________________________
>>>> > >Mailman-users mailing list -- mailman-users(a)mailman3.org
>>>> > >To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>>>> > >https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>>> >
>>>> >
>>>> > --
>>>> > Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>>>
>>>> --
>>>> thanks,
>>>> Abhilash Raj (maxking)
>>
>> --
>> thanks,
>> Abhilash Raj (maxking)
>>
>>
--
thanks,
Abhilash Raj (maxking)
6 years, 5 months
Re: [Mailman-Users] MM3 startup issues
by hansen＠rc.org
Mark Sapiro wrote:
> Restoring original Subject:, and doesn't this thread belong on
> mailman-users(a)msailman3.org - CCing there.
Sorry about both, Mark. I'm now here. Didn't know this one existed and was told to use the other list server.
> > On 1/4/20 5:47 PM, Allan Hansen wrote:
> > >
> > a. The server was set up with the wrong domain
> > name (our fault), so I had to add the ‘correct’ one. I had seen the command ‘Add Domain’
> > in
> > the UI and thought that would do it, but Brian kindly sent me a list of additional
> > instructions, which we then implemented. That made it possible to create a list in the new
> > domain, subscribe people and post messages. However, messages are not archived. What extra
> > step is needed to enable archiving?
> > What are the setytings in Postorius -> Settings -> Archiving?
> > Archive policy: Private archives
> > Archive archivers: hyperkitty, prototype
> Are the messages in Mailman's var/archives/prototype/LIST@DOMAIN/new
> directory?
Yes! I found them. But the message files end with the name of the old host, not the new host name under the new domain.
> Are there messages in Mailman's var/archives/hyperkitty/spool/ directory?
There are 5 .pck files there dated 1/5/2002.
> What's the setting for base_url in mailman-hyperkitty.cfg?
It's
base_url: https://<OLDHOSTNAME>/archives
I suppose it should be changed to
base_url: https://<NEWHOSTNAME>/archives
But:
What if we decided to have more than one domain, as the UI indicates is kosher?
> Are there any errors in var/logs/mailman.log?
Yep. Lots. They all are of this nature:
Exception in "hyperkitty" archiver
Traceback:
...
File "/opt/mailman/mm/venv/lib/python3.7/site-packages/urllib3/connection.py", line 430, in _match_hostname
match_hostname(cert, asserted_hostname)
File "/usr/lib/python3.7/ssl.py", line 327, in match_hostname
% (hostname, dnsnames[0]))
ssl.SSLCertVerificationError: ("hostname 'OLDHOSTNAME' doesn't match 'NEWHOSTNAME'",)
> > b. When I go to the server using the new domain (both
> > before and after logging in), I see not just the lists I created in the new domain, but
> > the lists created in the old domain. These lists of lists are domain specific and should
> > not be listed together. That’s just IMHO, but not serious, as I will soon get rid of
> > anything in the old domain. So feel free to ignore this one if this is intended, which I
> > could see might be the case.
> Set
> FILTER_VHOST = True
> in settings_local.py
> > Yes, Brian told me to do the same (well, settings.py).
> > This is the what I wrote back:
> > hansen@rccrmail:/opt/mailman/mm$ sudo vi settings.py
> > Edited the setting to:
> > # Only display mailing-lists from the same virtual host as the webserver
> > FILTER_VHOST = True
> You shouldn't edit settings.py. You should override the setting by
> putting it in settings_local.py.
Silly. I should have remembered that from back in the days. But if
the local file is overriding, it should have worked, I'd think.
> > Then Brian told me to restart qcluster:
> > hansen@rccrmail:/opt/mailman/mm$ sudo systemctl restart qcluster
> qcluster is not what you need to restart. You need to restart whatever
> is supporting the wsgi interface to Django. This may be uwsgi or
> gunicorn or ??. Whatever it is, that's what needs to be restarted.
Beats me, Mark. Brian, do you know?
> > I then logged in, but the same lists are showing. It
> > either takes the system a long time to reset or it has ignored the
> > setting.
> > Just tried it again, and the lists are all showing. Do I need to manually recompile the
> > file?
> >
> > c. I then
> > moved on to the templates and added a template for the message text to send to new
> > subscribers. That resulted in the following error message when I went in as moderator to
> > approve of a new subscription:
> > Something went wrong
> > Mailman REST API not available. Please start Mailman core.
> > Postorius is unable to access core's REST API. Is Mailman running?
> > Well, I was able to post to the lists, so I assume it is. Listing at processes, I
> > see a bunch of postgres, python3, nginx, django-adminp but
> > nothing explicitly with a ‘mailman’ string.
> If you do ps ax|grep runner=, what do you see? If you don't see at
> least three processes with --runner=rest:0:1, stop Mailman and start
> Mailman.
Yep. I did not get the whole string before. With your options I see three
of these rest:0:1 processes. So I guess the REST stuff is running anyway.
> If so, what's
> in /opt/mailman/mm/var/logs/mailmansuite.log or whatever it's
> called - defined as filename in the file handler in the LOGGING
> definition in settings(_local).py.
> hansen@rccrmail:/opt/mailman/mm/logs$ tail -f mailmansuite.log
> return Site.objects.get_current(request)
> File "/opt/mailman/mm/venv/lib/python3.7/site-packages/django/contrib/sites/models.py",
> line 60, in get_current
> return self._get_site_by_request(request)
> File "/opt/mailman/mm/venv/lib/python3.7/site-packages/django/contrib/sites/models.py",
> line 45, in _get_site_by_request
> SITE_CACHE[domain] = self.get(domain__iexact=domain)
> File "/opt/mailman/mm/venv/lib/python3.7/site-packages/django/db/models/manager.py",
> line 82, in manager_method
> return getattr(self.get_queryset(), name)(*args, **kwargs)
> File "/opt/mailman/mm/venv/lib/python3.7/site-packages/django/db/models/query.py", line
> 408, in get
> self.model._meta.object_name
> django.contrib.sites.models.Site.DoesNotExist: Site matching query does not exist.
> This is because of the domain change. In settings(_local).py you have a
> setting for SITE_ID. Possibly, setting that in settings_local.py to a
> value one greater that the current setting will do.
Trying setting it to 1 (it was 0 before).
> To be certain of the
> correct value, you need to look at the django_site table in mailman's
> database. That table has 3 columns, id, domain and name. The id for the
> row of your domain is what needs to be set as SITE_ID.
You lost me here. No idea how to access the Mailman database.
Brian, do you know?
> > [ABH] At this point I click to accept a subsription
> > request:
> > ERROR 2020-01-06 07:32:41,439 699 postorius.middleware Mailman REST API not available
> > Traceback (most recent call last):
> > File "/opt/mailman/mm/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line
> > 672, in urlopen
> > chunked=chunked,
> > File "/opt/mailman/mm/venv/lib/python3.7/site packages/urllib3/connectionpool.py", line
> > 421, in _make_request
> > six.raise_from(e, None)
> > File "<string>", line 3, in raise_from
> > File "/opt/mailman/mm/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line
> > 416, in _make_request
> > httplib_response = conn.getresponse()
> > File "/usr/lib/python3.7/http/client.py", line 1321, in getresponse
> > response.begin()
> > File "/usr/lib/python3.7/http/client.py", line 296, in begin
> > version, status, reason = self._read_status()
> > File "/usr/lib/python3.7/http/client.py", line 265, in _read_status
> > raise RemoteDisconnected("Remote end closed connection without"
> > http.client.RemoteDisconnected: Remote end closed connection without response
> > So yes, the REST API is not there. How can I get it going?
> Stop Mailman and then start it.
No idea here, either. I looked online and found this:
$ from mailman.commands.cli_control import Stop
But it complained:
from: too many arguments
Must have found an out-of-date page. :-)
Yours,
Allan
5 years, 6 months
Re: No styling for web apps?
by Morris Jones
On 11/26/20 10:09 PM, Mark Sapiro wrote:
> On 11/26/20 8:44 PM, Morris Jones wrote:
>> I have most of a new mailman suite server built and working ...
>>
>> collectstatic gathered a good array of static files (uncompressed so far)
>>
>> nginx appears to be serving them successfully
>>
>> The Firefox developer tools show css and js files all being delivered
>>
>> But none of the postorius or hyperkitty pages exhibit any styling at
>> all, only bare HTML elements.
>>
>> The Firefox inspector will show the available styles being requested and
>> delivered (style.css, bootstrap, jquery, all the things).
>>
>> But if I use the inspector to look at the page, nothing has any CSS
>> properties.
>>
>> I know I'm missing something simple.
>
> Try running collectstatic and, if you use it, compress again. These have
> to be run following various kinds of changes.
>
Appreciated, but to no avail:
(venv) mailman@mail:~/mailman-suite/mailman-suite_project$ python
manage.py collectstatic
You have requested to collect static files at the destination
location as specified in your settings:
/opt/mailman/mailman-suite/mailman-suite_project/static
This will overwrite existing files!
Are you sure you want to do this?
Type 'yes' to continue, or 'no' to cancel: yes
0 static files copied to
'/opt/mailman/mailman-suite/mailman-suite_project/static', 399 unmodified.
(venv) mailman@mail:~/mailman-suite/mailman-suite_project$ python
manage.py compress
CommandError: Compressor is disabled. Set the COMPRESS_ENABLED setting
or use --force to override.
And then https://mailman.whiteoaks.com
I'll spend time in the Firefox developer tools to see if I can figure
out what's missing. A DuckDuckGo search suggested it could be a
character encoding issue. The style sheets all come up reporting "0
rules" in the developer tools. I haven't tried any other browsers either.
Best regards,
Mojo
--
Morris Jones, Monrovia, CA
BridgeMojo <http://bridgemojo.com>
Old Town Sidewalk Astronomers <http://otastro.org>
Mojo's Blog <http://mojo.whiteoaks.com>
4 years, 8 months
Re: Custom Archiver
by Mark Sapiro
On 4/1/23 16:33, Peter Knowles wrote:
> Mark,
>
> The whole reason I'm developing my own "frontend" for both "list management" and "archive" access is because Python isn't really the language I speak... I'm a PHP guy, and so I'm working to build with the language I know and love, while utilizing Mailman for what it does best... In addition to implementing the frontend the way I want it (expose/hide feature that are useful/unuseful to our users), I want to fully customize the visual appearance to represent the SaaS offering properly.
This is exactly the reason why Mailman core was decoupled from archiving
and the web UI. HyperKitty and Postorius were developed because we knew
that almost no one would use Mailman core without archiving and a web
management UI, but we always hoped that others might develop and
contribute other web based archiving and management interfaces.
Brian Carpenter of EMWD who tragically died of COVID did exactly that
with his Affinity and Empathy PHP applications that interface with
Mailman core via the REST API. EMWD hosts lists that use those
interfaces, and had Brian not died, some version of those might be open
sourced, but his initial thrust was to keep them proprietary for his
business reasons. There is an early description of Brian's concept for
Affinity at https://gitlab.com/mailman/affinity.
> *** we're not try to hide that Mailman is in the equation, just how the overall interface is visually represented ***
>
> That all being said, on a slight segway... Aside from using the REST API, is it practical or possible to manipulate the database directly? That is, in doing so would it just work or are there extra moving parts when an API request is issued?
It probably is possible, but I would definitely discourage it. The REST
API should provide everything you need without your having to
reimplement everything it does.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
2 years, 4 months
Re: Importing config and archives only partially succeeded?
by Eric Broens
I just notice in the webserver logfile, the same request for the same mlist, msgid and key several times per second for about 3 hours now:
"GET /archives/api/mailman/urls?mlist=....&msgid=...&key=... HTTP/1.1" 200 105 "-" "python-requests/2.25.0"
The request seems to originate from the server where the mailman suite is installed
Best Regards,Eric
On Sunday, December 13, 2020, 01:54:54 AM GMT+1, Eric Broens via Mailman-users <mailman-users(a)mailman3.org> wrote:
The settings - Archiving, are not different from the ones that do show the messages:"Private Archives" selected, and Active archivers "hyperkitty" also selected.
I was trying to find out how the hyperkitty page gets the mails from the database, and if I can start from there.Or maybe something is not completely correct in the database tables.
Best Regards,Eric
On Sunday, December 13, 2020, 01:27:53 AM GMT+1, Mark Sapiro <mark(a)msapiro.net> wrote:
On 12/12/20 3:56 PM, Eric Broens via Mailman-users wrote:
> After everything was imported, and I noticed that the mails were not shown in the archives for half of the mailinglists, I ran "manage.py update_index" to index the mails for all mailinglists.This command is still running for hours.Since it builds the search index, I don't expect that this would make the mails available on the webpage.
I agree, that won't make the mails available on the webpage.
If you're using the default Whoosh search engine, "hours" is not
unexpected. Elasticsearch is much faster and Xapian faster still.
What do you see in Postorius - Settings -> Archiving for the lists whose
messages don't show?
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
_______________________________________________
Mailman-users mailing list -- mailman-users(a)mailman3.org
To unsubscribe send an email to mailman-users-leave(a)mailman3.org
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
_______________________________________________
Mailman-users mailing list -- mailman-users(a)mailman3.org
To unsubscribe send an email to mailman-users-leave(a)mailman3.org
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
4 years, 7 months
Re: Unable to download attachments from HyperKitty
by Abhilash Raj
On Tue, Feb 19, 2019, at 7:32 AM, Derek Lambert wrote:
> I’m having an issue downloading attachments from HyperKitty. The
> download begins as expected, but never progresses. Eventually it times
> out. I imagine the browser is expecting data matching the
> content-length header, but never receives it.
>
> Inspecting the contents of the download file reveals (address varies):
>
> <memory at 0x7f148fcb9588>
>
> I’ve been able to work around the issue with the following patch:
>
> ---
> env/lib/python3.7/site-packages/hyperkitty/models/email.py.orig 2018-12-06 21:38:39.544448735 +0000
> +++
> env/lib/python3.7/site-packages/hyperkitty/models/email.py 2018-12-06
> 23:01:39.923239421 +0000
> @@ -309,7 +309,7 @@
> def get_content(self):
> folder = self._get_folder()
> if folder is None:
> - return self.content
> + return self.content.tobytes()
> filepath = os.path.join(folder, str(self.counter))
> if not os.path.exists(filepath):
> logger.error("Could not find local attachment %s for email
> %s",
>
> I haven't been able to replicate the issue with the HyperKitty tests,
> presumably the issue is somewhere in the Django <-> uwsgi <-> nginx
> stack.
>
> My configs are based on mailman-suite and maxking/docker-mailman. I've
> tried stripping out all the performance tweaks from my uwsgi and nginx
> configs, but no change.
>
> Versions:
> HyperKitty 1.2.1
> Postorius 1.2.3
> Django 2.1.7
> nginx 1.14.1
> uwsgi 2.0.18
> Python 3.7.2
>
> Has anyone come across this before?
I am not sure if this is exactly a bug in Hyperkitty. I am not able to replicate this with a extremely complicated test: Downloading a signature file in an email on lists.mailman3.org. It runs on gunicorn + Django 2.1.5 ( I think) and latest Git heads of respective projects.
Is this a recurring or one time thing that you observed?
It would be great if we could you could capture the HTTP Headers with and without the patch you sent above to help debug why does `.tobytes()` lets you download.
--
thanks,
Abhilash Raj (maxking)
6 years, 5 months
Re: Hyperkitty import from mailman2
by Sten Aus
Oh my god. :) That's it! There are no latest messages and front page
shows only last month "activity". Thank you! I did not thought that I
should check some older posts. :D
So, import is working just fine, I guess! ;)
But how did you manage to import all the lists and members from Mailman2
to Mailman3? As I understand Mailman3 does not "support" (by manual)
import right now?
On 24/12/2017 00:11, Marvin Gülker wrote:
> Just a suggestion -- if your list didn't have any e-mails in the last
> month, the front page will show it as inactive. Dig into the archive
> links on the left side to actually see your e-mails. I've been bitten by
> that behavior as well. The front page simply doesn't show e-mails older
> than a month.
>
> Look here: https://lists.secretchronicles.org/hyperkitty/list/tsc-devel@lists.secretch…
>
> No emails on the front page. Now go here:
> https://lists.secretchronicles.org/hyperkitty/list/tsc-devel@lists.secretch…
>
> Well there.
>
> Greetings
> Marvin
>
> Am 23. December 2017 um 01:05 Uhr +0200 schrieb Sten Aus <sten.aus(a)eenet.ee>:
>> Hi
>>
>> Thank you Mark and Simon for your input!
>>
>> I can see imported list from Hyperkitty GUI, but it's inactive.
>> I can confirm that I can see those emails from command line (python shell
>> shows Email Objects).
>>
>> Import shows everything is OK:
>>
>> # python2 manage.py hyperkitty_import --settings settings --ignore-mtime -v
>> 3 --since 2010-01-01 -l test(a)list.domain.com /tmp/test.mbox
>> Only emails after 2010-01-01 00:00:00+02:00 will be imported
>> Importing from mbox file /tmp/test.mbox to test(a)list.domain.com
>> /--some 22 emails being imported here --/
>> 22 emails read
>> 22 email added to the database
>> 22 emails are stored into the database
>> Computing thread structure
>> Synchronizing properties with Mailman
>> Warming up cache
>> The full-text search index will be updated every minute. Run the 'manage.py
>> runjob update_index' command to update it now.
>>
>> # python2 manage.py update_index --settings settings
>> Indexing 50 emails
>> NB! Only 6 of them are archived from new list (which has been created from
>> Apache). I have imported 2 times 22 emails from /tmp/test.mbox file (to
>> different lists).
>>
>>
>> I am using Apache2.4 with mod wsgi and Postorius and Hyperkitty are in the
>> same django application.
>> Also tested with running server directly and then connecting to a port 8000,
>> but I guess it does not matter, because both of them are using Python
>> script.
>>
>> I have not yet managed to set up cron tasks, as this is relatively new
>> installation, but I have ran them manually. Also, I have done update_index
>> (runjob update_index and manage.py update_index, both ways).
>>
>>
>> SQLite database shows emails, for example there's a thread
>> FMXQVFDRQVRMINVXI4DJB4SHYTJ3EPWP, but If I insert it to web URL I get Django
>> error: No Thread matches the given query.
>>
>>
>> As Postorius and Hyperkitty are in the same Django, they share sqlite
>> database (mailman/web/database.db). Mailman has it's own database in
>> mailman/var/data/mailman.db.
>>
>>
>> Happy holidays
>> Sten
>>
>>
>> On 22/12/2017 21:09, Simon Hanna wrote:
>>> On 12/22/2017 03:39 PM, Sten Aus wrote:
>>>> Hi
>>>>
>>>> I have installed mailman3 - it looks very great!
>>>>
>>>> Now I wanted to try import from mailman2 mbox, so i did:
>>>>
>>>> python2 manage.py hyperkitty_import --settings settings
>>>> --ignore-mtime -v 3 --since 2010-01-01 -l test(a)list.domain.com
>>>> /tmp/test.mbox
>>>>
>>>> It shows that it have sucessfully imported all emails and I can see
>>>> them from database (sqlite3), but I cannot see them from GUI
>>>> (hyperkitty). Only thing I can see is this inactive "test" list.
>>>>
>>> How did you try to access them using Hyperkitty? Can you try doing that
>>> while logged in as a django superuser? Could you try using a permalink
>>> for a message? If you can access the files from the database you should
>>> be able to construct the urls using the list name and the
>>> message_id_hash of a message. You should try this using DEBUG=True in
>>> the settings.py. The URL would be similar to http://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/…
>>>
>>>
>>> Another option would be to check if you can access the data using a
>>> python shell.
>>>
>>> `python manage.py shell` to open a python shell preconfigured for
>>> Hyperkitty and then input the following two lines:
>>>
>>> from hyperkitty.models.email import Email
>>> Email.objects.all()
>>>
>>>
>>> This should print out all the messages that are stored in the database.
>>> (You can limit them by appending "[10:]", or any other python slice)
>>>
>>> What do you see in the UI? Is it somewhat similar to
>>> http://lists.mailman3.org/archives/list/mailman-users@mailman3.org ?
>>> _______________________________________________
>>> Mailman-users mailing list
>>> mailman-users(a)mailman3.org
>>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>> _______________________________________________
>> Mailman-users mailing list
>> mailman-users(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
7 years, 7 months
Re: Mailman installation results in KeyError: 'env_py'
by Abhilash Raj
On Sun, Oct 22, 2017, at 09:36 AM, Marvin Gülker wrote:
> Am 22. October 2017 um 07:49 Uhr -0700 schrieb Mark Sapiro
> <mark(a)msapiro.net>:
> > or you can install the fix at
>
> Thanks. Applying this patch makes it work. Now I've got a different
> problem, though. I can't get it to connect to Postfix for sending
> emails it appears. After I created a superuser account using
>
> $ python manage.py createsuperuser
>
> as described on <http://postorius.readthedocs.io/en/latest/setup.html>,
> I tried to login using this credentials into Postorius. It then reported
> it sent a confirmation email to me (for an account I created on
> commandline?!), but Postfix' logs reveal that it didn't; there's not
> even a sign of an attempted delivery in /var/log/mail.log. I've
> configured the Postfix configuration in mailman.cfg like this:
>
> [mta]
> incoming: mailman.mta.postfix.LMTP
> outgoing: mailman.mta.deliver.deliver
> lmtp_host: 127.0.0.1
> lmtp_port: 8024
> smtp_host: localhost
> smtp_port: 25
> configuration: python:mailman.config.postfix
>
> as was outlined on
> <http://mailman.readthedocs.io/en/latest/src/mailman/docs/mta.html>.
>
> mailman-suite's settings.py contains:
>
> # Change this when you have a real email backend
> EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
> EMAIL_HOST = 'localhost'
> EMAIL_PORT = 25
>
> which was described on <http://docs.list.org/en/latest/config-web.html>.
>
> For completeness, the following is in Postfix' main.cf, but is not the
> cause of the problem as it affects the incoming direction, whereas I
> refer to the outgoing one:
>
> transport_maps = hash:/home/mailman/var/data/postfix_lmtp
> local_recipient_maps = hash:/home/mailman/var/data/postfix_lmtp
>
> I failed to locate any kind of logging information about the failed
> email, so I don't even know where I should start looking for the
> problem. var/logs/smtp.log is empty in the mailman directory, and
> likewise is logs/mailmansuite.log in the mailman-suite_project
> directory.
As Simon mentioned, try using DEBUG=False, if you don't already have
that. With DEBUG=True, you should see emails printed in a `emails`
directory under `mailman-suite_project` directory.
This is a confirmation email to verify the email addresses, which
happens even if you create an account from command line.
> That being said, the enormous number of configuration files to consider
> and dozens of cross-references to other pages in the docs make the
> installation guide hard to follow; it also lacks logical structure in my
> opinion, when it refers to parts of the configuration which are later
> explained. For someone like me who never programmed a Django application
> (Ruby dev) this is all pretty cryptic. Especially, that some parts of
> information are on <http://docs.list.org/en/latest/index.html> whereas
> other parts are scattered over readthedocs.io makes it very
> complicated.
There are only two primary configuration files (there are a few more for
finer configurations ;-):
- mailman.cfg: This is the configuration for the Core engine and can be
located any of the places mentioned in [1] or pointed to by environment
var MAILMAN_CONFIG_FILE.
- settings.py: This is the configuration file for Django under
`mailman-suite_project` and controls the frontend (Postorius &
Hyperkitty). Django is a huge framework with literally 1000s of options.
It would be impossible to copy that information from Django's
documentation and keep it updated in our docs. Hence, we have links to
appropriate places when needed. Same goes out for the rest of the
libraries that we use.
All the settings mentioned in the documentation go to either of these
files, except, when configuring web server and mail server. Please open
an issue[2] if it is not clear where a particular configuration snippet
goes and we will fix it.
[1]: http://docs.mailman3.org/en/latest/config-core.html
[2]: https://gitlab.com/mailman/mailman-suite-doc/issues
Hope that helps!
--
Abhilash Raj
maxking(a)asynchronous.in
7 years, 9 months
Re: Installing Mailman3 on Debian Bullseye
by Stephen J. Turnbull
Franklin Weng writes:
> Thanks for your comments!
Thank you for following up on my queries.
> > As far as I know, Debian packages use the system Python and the
> > system Django. Is that correct?
>
> Correct.
This means that you're stuck with the Debian version of the Mailman 3
suite until they update which can be slow. Mailman 3 is now mature
enough that if you're still happy with the Debian version a month
after installing, you'll probably still be happy with it when Debian
updates. It's a good tradeoff for the majority.
But for example mixing a "testing" Mailman with a "stable" Python is
likely to end badly because we frequently upgrade required levels of
dependencies. So you either have to get a Debian upgrade of Mailman
of your current Debian version, or you need to move to a whole
not-stable Debian system. Not a pleasant choice if you think you need
new features or a bugfix *now*.
This is why we recommend the venv or other isolated installations.
They allow you to upgrade the dependencies as well as the core
packages without interfering with the rest of your system.
> IIRC I got some errors about handling characters encoding if I
> don't set up charset in language.master and language.en
> section. But the memory is vague and I don't want to go back
> retesting that...
I suspect that you set up charset in .master and realized that it
doesn't work because .en overrides it with ASCII, and then fixed .en.
That's what happened to me.
> Honestly I was totally confused by all these, and as I have said,
> the MM3/Django/Hyperkitty/... documents didn't help much.
Upstream documents are rarely very helpful with whole sets of large
packages from distros in my experience. You install the packages, you
go through a bunch of configuration screens, and you finish knowing
*nothing about how anything works*. Then you need a little tweak that
the configuration screens don't deal with, and you don't even have the
vocabulary to grep or Google effectively. :-( Then again, I'm the
kind of person who prepares to sleep by reading software manuals. ;-)
But I do think you lose quite a bit of knowledge by using packages,
and it's painful to find obscure details without it.
Django doesn't actually show its face much in Mailman, so Django docs
are not much use even to the site admin, unless you actually hack on
Mailman. The only exceptions are authentication and the template
language. Authentication is just hard, with more details then the
Lego set in the Lego store. The templating docs are pretty good, IMO.
The Mailman docs are in somewhat sad state. The content is actually
quite comprehensive and detailed as these things go, but much of it is
hard to discover. The original idea was to keep the docs in the
sources, as doctests for examples and in the docs directory for
exposition. These docs are more developer-oriented, obviously. Then
there were several attempts to create a suite package which would
require core, Postorius, and HyperKitty and streamline the setup
process. The "mailman-suite" version came with a companion
"mailman-suite-docs" package, which I think is now the basis for
mailman.readthedocs.io aka docs.mailman3.org. However mailman-suite
turned out to more trouble than it was worth, and mailman-web was
developed to address the more useful approach of unifying some
mailman-to-Django APIs for Postorius and HyperKitty (the
django_mailman3 package) and configuration (a unified settings.py, as
well as standardized top-level urls.py and views.py). Of course each
of these contributed more documentation, some of it redundant, older
versions left to slowly become obsolete (ie, incorrect) ....
"Someday" I'd like get the documents reorganized, eliminate as much
redundancy as possible and make things more discoverable. But it's a
mammoth task.
Steve
1 year, 11 months
Deactivate list without deleting it?
by Colin Watson
Hi,
I'm experimenting with a port of Launchpad's mailing list infrastructure
to Mailman 3. If Barry's reading, he may remember that this is no
trivial task, so I'm very definitely not promising that it will be done
any time soon. :-) However, for the most part it looks relatively
tractable with the aid of the plugin improvements in Mailman 3.2.0; I
have the rough outline of a plugin that implements much the same feature
set as our current pile of monkey-patches against Mailman 2.1 in a much
nicer way.
One of the major things I haven't figured out yet is list deactivation
and reactivation. In our current 2.1-based code, you can deactivate a
list in a reversible way, which basically tars up the list directory so
that the list archives can be restored later. This isn't a native
operation in 2.1, but it could be done because everything lived on the
filesystem. In 3, it's less trivial. We could go to some effort to
dump things out of the HyperKitty database and restore it on
reactivation, but that seems rather complex and a bit silly.
Ideally, I'd like mailing lists to have a "deactivated" state in
addition to full deletion. In this state, the list would appear
nonexistent for most purposes: incoming email would be rejected,
Postorius would deny its existence, etc. There'd need to be
corresponding extensions to the IArchiver interface which caused the
archiver to put its archives of that list into a corresponding
deactivated state, in which it would retain the messages but queries for
them would return 404 or similar; these extensions would need to be
optional, since while it should be relatively easy to do this in
HyperKitty it could be hard or impossible for other archivers, and
perhaps this should be a separate operation from deactivating the list
itself since people may want to close a list while leaving its archives
accessible.
Has there been any discussion of this in the past, or any advice as to
implementation strategy?
Thanks,
--
Colin Watson [cjwatson(a)ubuntu.com]
7 years, 6 months