On Mar 30, 2017, at 12:34 AM, David Krantz wrote:

When starting this as the recipe seems reasonable I find that I have no reliable source for git links for the parts. Some of the git links I find are marked with "do not use" so it is a bit hard having confidence in selecting one of the others.

All the Mailman 3 suite subprojects are listed here:

https://gitlab.com/mailman

Those project links will take you to the active git repositories. The only one you should not use is mailman-bundler.

Releasing 3.1 and a better deployment story are top priorities for the Pycon sprints. I wish we'd been able to release 3.1 before Pycon, but it just seems unlikely. Plus we have a fine tradition to uphold. :)

-Barry

On Thu, Mar 30, 2017 at 1:06 AM, Mark Sapiro <mark@msapiro.net> wrote:

To be more specific, the projects that are required for a fully functioning Mailman installation with Postorius and Hyperkitty are

Mailman (the core)

Hyperkitty (the archiver)

Postorius (the web list management UI)

django-mailman3 (common django things used by Postorius and Hyperkitty)

MailmanClient (python bindings for the core REST API used by Postorius and Hyperkitty)

HyperKitty - Mailman plugin (plugin for the core to communicate with Hyperkitty)

I tried to upgrade today. The result is a broken install which fails when trying to upgrade the database.

$> ./bin/mailman-post-update [...] File "/opt/mailman/mailman-bundler/eggs/Django-1.8.17-py2.7.egg/django/db/backends/base/base.py", line 236, in _commit return self.connection.commit() File "/opt/mailman/mailman-bundler/eggs/Django-1.8.17-py2.7.egg/django/db/utils.py", line 94, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/opt/mailman/mailman-bundler/eggs/Django-1.8.17-py2.7.egg/django/db/backends/base/base.py", line 236, in _commit return self.connection.commit() django.db.utils.IntegrityError: update or delete on table "auth_user" violates foreign key constraint "social_auth_usersocial_user_id_193b2d80880502b2_fk_auth_user_id" on table "social_auth_usersocialauth" DETAIL: Key (id)=(3) is still referenced from table "social_auth_usersocialauth".

I have tried to drop and recreate the databases but I get the above result anyway.

The description in https://wiki.list.org/DOC/Mailman%203%20installation%20experience is very hazy. The process of symlinking in libraries is fiddly to say the least. I get mailman to start but not any web interface. I have not tested mailman so if it works rather than just start I do not know.

There is a description for a clean install for development at http://docs.mailman3.org/en/latest/devsetup.html, but I suspect that it isn't usable for anything other than development. I have read it parallell to try to interpret the installation experience description which in turn describes more of what I am trying to do. No luck though.

regards // David

On Fri, Mar 31, 2017 at 12:01 AM, Mark Sapiro <mark@msapiro.net> wrote:

This is <https://gitlab.com/mailman/hyperkitty/issues/106>.

I'm not sure what you mean by drop and recreate the databases. You should just drop the social_auth* tables, or if there's nothing in the database you want, just starting with an empty database should work.

Very good! Thank you very much, I got a bit further.

The database contained a couple of test lists that I can recreate, so nothing of value. The strange thing was that drop database followed by create database did not yield an empty database in Postgres. I probably forgot a commit somewhere.

I got a bit further. The problem with not being an expert on Django is that the configuration of all web frameworks is that is a bit involved. I have no experience of Django outside of this and I just want to get it running in this case.

Now I get the following errors in the mailman_web-log, somehow complaining on the url mapping.

ERROR 2017-03-31 00:51:31,558 exception 2456 140113416898304 Internal Server Error: /mailman3/lists/ Traceback (most recent call last): [...] File "/opt/mailman/mailman-bundler/eggs/Django-1.8.17-py2.7.egg/django/template/defaulttags.py", line 458, in render url = reverse(view_name, args=args, kwargs=kwargs, current_app=current_app) File "/opt/mailman/mailman-bundler/eggs/Django-1.8.17-py2.7.egg/django/urls/base.py", line 91, in reverse return force_text(iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs))) File "/opt/mailman/mailman-bundler/eggs/Django-1.8.17-py2.7.egg/django/urls/resolvers.py", line 497, in _reverse_with_prefix raise NoReverseMatch(msg) NoReverseMatch: Reverse for 'hk_user_login' not found. 'hk_user_login' is not a valid view function or pattern name.

The Django 1.18 points to a Django 1.11rc1, should probably be mentioned.

My urls.py (a copy from the docs, I think)

from django.conf.urls import include, url from django.core.urlresolvers import reverse_lazy from django.views.generic import RedirectView from django.contrib import admin

# Comment the next two lines to disable the admin: #admin.autodiscover()

urlpatterns = [ url(r'^$', RedirectView.as_view(url=reverse_lazy('hyperkitty.views.index.index'))), url(r'^mailman3/', include('postorius.urls')), url(r'^archives/', include('hyperkitty.urls')), url(r'', include('django_mailman3.urls')), url(r'^accounts/', include('allauth.urls')), # Django admin url(r'^admin/', include(admin.site.urls)), ]

cheers

// David

On 03/30/2017 04:45 PM, David Krantz wrote:

Never mind, now I got it to work. Some kind of intermittent server error (500) left but I guess that's solvable too. Thanks for all your help on this list!

Cool.

We're really working on getting a consistent, workable installation story and even getting some packaged solutions. I know we're not yet there, but we're here to help when people run into problems.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 03/31/2017 08:10 AM, David Krantz wrote:

On Fri, Mar 31, 2017 at 3:13 AM, Mark Sapiro <mark@msapiro.net> wrote:

...

We're really working on getting a consistent, workable installation story and even getting some packaged solutions. I know we're not yet there, but we're here to help when people run into problems. The site now works, but not the / url for the site. I get this error in the mailman-web.log (with a very long stacktrace before it)

NoReverseMatch: Reverse for 'hyperkitty.views.index.index' not found. 'hyperkitty.views.index.index' is not a valid view function or pattern name.

I think that would correspond to this line in urls.py

url(r'^$', RedirectView.as_view(url=reverse_lazy('hyperkitty.views.index.index'))),

Yes, that's the culprit. Replace it with the following:

url(r'^$', RedirectView.as_view(url=reverse_lazy('hk_root'))),

You can also update whatever guide you were using...

but this corresponds exactly to what the example config says and I know of no other name for hyperkitty. Probably it is dependent upon understanding the url dispatching in Django but that takes some time to read up on. The interesting thing is that /archives works so it is just this redirect that has problems. I can probably get apache to do the redirect instead but that is not the most elegant solution.

Then I of course have things to do on getting social logins to work. I currently get "DoesNotExist: SocialApp matching query does not exist." if trying to connect an account to google (which I started with). Not a showstopper as passwords do work. Did you follow django-allauth's guide on setting them up?

On 03/31/2017 01:06 AM, David Krantz wrote:

On Fri, Mar 31, 2017 at 9:29 AM, Simon Hanna <simon.hanna@serve-me.info> wrote:

...

On 03/31/2017 08:10 AM, David Krantz wrote:

...

I think that would correspond to this line in urls.py

url(r'^$', RedirectView.as_view(url=reverse_lazy('hyperkitty.views.index.index'))),

Yes, that's the culprit. Replace it with the following:

url(r'^$', RedirectView.as_view(url=reverse_lazy('hk_root'))),

You can also update whatever guide you were using...

Great, will try that! I don't know whether I can update the guide. It's the one referenced earlier in this thread named something like setup experience of mailman 3 in production.

Yes, that comes from "https://wiki.list.org/DOC/Mailman%203%20installation%20experience", specifically the attachment urls.py.

The interesting thing is that works for me both at <https://lists.mailman3.org/> and <https://mail.python.org/mm3/>.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 04/01/2017 08:19 AM, Simon Hanna wrote:

On March 31, 2017 6:34:18 PM GMT+02:00, Mark Sapiro <mark@msapiro.net> wrote:

...

The interesting thing is that works for me both at <https://lists.mailman3.org/> and <https://mail.python.org/mm3/>.

Are you using the latest version of django?

No, I'm using Django 1.9.9. I suppose that means it will quit working if I upgrade to Django 1.10.

So, I updated my urls to

url(r'^$', RedirectView.as_view(url=reverse_lazy('hk_root'))),

and updated the attached files at <https://wiki.list.org/DOC/Mailman%203%20installation%20experience>.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 04/09/2017 12:23 PM, David Krantz wrote:

Now everything works for me except the archiver. There I get strange errors where it requires authentication. I _have_ set MAILMAN_ARCHIVER_KEY (in production.py) and api_key in mailman_hyperkitty.conf to the same value within single quotes. This is probably something extremely silly that my jetlagged brain cannot see.

In mailman_hyperkitty.conf, the key isn't quoted. I.e., if in production.py or wherever, you have set

MAILMAN_ARCHIVER_KEY = 'arch_secret'

in mailman_hyperkitty.conf you set

[general] ... api_key: arch_secret

No quotes around the api_key value.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On Mon, Apr 10, 2017 at 1:06 AM, Mark Sapiro <mark@msapiro.net> wrote:

In mailman_hyperkitty.conf, the key isn't quoted. I.e., if in production.py or wherever, you have set

MAILMAN_ARCHIVER_KEY = 'arch_secret'

in mailman_hyperkitty.conf you set

[general] ... api_key: arch_secret

No quotes around the api_key value.

Ok, I'll try that. Thought that I had, but probably not just that combination then.

Also: Found a bug in Hyperkitty, or at least in my setup: Private lists are visible in the "By Name" dialog in the archives when not logged in. The actual content is not visible, but the activity statistics is displayed (although there is none as the login for archiving as of yet does not work). Hopefully this will correct itself when the communication mailman3-hyperkitty gets active.

cheers // David

On 1/3/19 11:00 AM, kelly.close@lrewater.com wrote:

Hi David,

Did you ever find a resolution to these Authorization errors? I'm experiencing the same problem in a new setup. Would love to know if you figured it out!

The code that produces that error is:

    if request.GET.get("key") != settings.MAILMAN_ARCHIVER_KEY:
        return HttpResponse(
            """<html><title>Auth required</title><body>
            <h1>Authorization Required</h1></body></html>""",
            content_type="text/html", status=401)

I.e., the key in the request which comes from the

[general] api_key:

setting doesn't match the key set as MAILMAN_ARCHIVER_KEY. Do you have quotes around the api_key setting? That would be a mistake. See <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/T7N7NNZYROQ3Z2IGPNRTHWTOEDYJZOY7/>

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 1/3/19 5:33 PM, kelly.close@lrewater.com wrote:

Thanks Mark. I keep coming back to this and checking it again and again. The two keys match, and there are no quotes on the api_key.

This is what I thought the problem was from the start but I just can't see anything wrong with those two settings. I'm going to look again - maybe entirely re-type the lines - what if there is a trailing space or something...

I don't think trailing white space on the api_key setting is significant, and whitespace following the trailing quote in the MAILMAN_ARCHIVER_KEY setting is not significant, but embedded whitespace in the key itself could be an issue.

Possibly Mailman and/or Hyperkitty aren't getting this info from the files you think they are. The command that you use to start Mailman should have a -C option pointing to Mailman's config. That in turn has a [archiver.hyperkitty] section pointing to the Mailman Hyperkitty config. That in turn contains the api_key.

Then whatever process you use to run the wsgi application will set DJANGO_SETTINGS_MODULE in it's environment to point to settings.py or whatever you call it (without the .py extension) and that in turn imports overrides from settings_local.py. You need to understand what the working directory is of the process that imports the DJANGO_SETTINGS_MODULE.

If you need more info, you could try modifying hyperkitty/hyperkitty/views/mailman.py at around lines 64 - 68 to:

    if request.GET.get("key") != settings.MAILMAN_ARCHIVER_KEY:
        return HttpResponse(
            """<html><title>Auth required</title><body>
            <h1>Authorization Required</h1>
            {} != {}</body></html>"""
            .format(request.GET.get("key"),
                    settings.MAILMAN_ARCHIVER_KEY),
            content_type="text/html", status=401)

This should display the values that don't match as part of the error response. Note that this is Python and leading whitespace is significant and if you're reading this in HyperKitty, you won't see it unless you "display in fixed font" (the "A" icon), and even then, there may be line wrap.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

(sorry - that should be MAILMAN_ARCHIVER_KEY)