Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: Search problem in 3.2.1?

by Mark Sapiro

On 10/22/19 7:51 AM, Mark Dadgar wrote: > Anyone seen this before? This is the result of any search query I run. I installed ElasticSearch as the search engine yesterday and this is the result. It is still indexing (large archives). How did you install ElasticSearch. What ElasticSearch version? This list uses elasticsearch 5.5.3 installed from the .deb as described at <https://www.elastic.co/guide/en/elasticsearch/reference/5.5/deb.html#instal…>. You can't use a later version because there are no python bindings for versions later than 5.x In addition we use the python bindings installed with `pip install elasticsearch5` <https://pypi.org/project>/elasticsearch5/ This version also requires haystack > 2.8.1 which we installed via `pip install -e git+https://github.com/django-haystack/django-haystack.git@master#egg=djang… Finally, we have these patches to the python bindings > --- elasticsearch5_backend.py.orig 2019-05-24 21:26:04.031867202 -0700 > +++ elasticsearch5_backend.py 2019-05-25 08:45:42.521374647 -0700 > @@ -28,6 +28,15 @@ > installation of 'elasticsearch>=5.0.0,<6.0.0'. \ > Please refer to the documentation." > ) > +# We need to override some elasticsearch_backend settings as string type is > +# gone in elasticsearch 5. > +from haystack.backends import elasticsearch_backend > +elasticsearch_backend.DEFAULT_FIELD_MAPPING = { > + "type": "text", "analyzer": "snowball"} > +elasticsearch_backend.FIELD_MAPPINGS["edge_ngram"] = { > + "type": "text", "analyzer": "edgengram_analyzer"} > +elasticsearch_backend.FIELD_MAPPINGS["ngram"] = { > + "type": "text", "analyzer": "ngram_analyzer"} > > > class Elasticsearch5SearchBackend(ElasticsearchSearchBackend): > --- elasticsearch_backend.py.orig 2019-06-16 10:21:42.441334321 -0700 > +++ elasticsearch_backend.py 2019-06-16 10:24:20.802264202 -0700 > @@ -78,8 +78,10 @@ > ) > > # Settings to add an n-gram & edge n-gram analyzer. > + # And, set max_result_window. > DEFAULT_SETTINGS = { > "settings": { > + "max_result_window": "100000", > "analysis": { > "analyzer": { > "ngram_analyzer": { The patch to elasticsearch_backend.py may require an even bigger value for max_result_window. The issue is the monthly Django hyperkitty - update_and_clean_index job runs into <https://stackoverflow.com/questions/41677198/result-window-is-too-large-fro…> > Mailman 3.2.1 from the debian packages in Ubuntu 19.10. > > - Mark > ----- > mark(a)pdc-racing.net | 408-348-2878 > > > Begin forwarded message: > > From: root(a)pdc-racing.net > Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/search > Date: October 22, 2019 at 7:44:49 AM PDT > To: mark(a)pdc-racing.net > > Internal Server Error: /mailman3/hyperkitty/search > > UnboundLocalError at /hyperkitty/search > local variable 'search_exception' referenced before assignment > > Request Method: GET > Request URL: https://mail.pdc-racing.net/mailman3/hyperkitty/search?q=Ken ... > Traceback: > > File "/usr/lib/python3/dist-packages/django/core/paginator.py" in count > 79. return self.object_list.count() > > File "/usr/lib/python3/dist-packages/haystack/query.py" in count > 522. return len(self) > > File "/usr/lib/python3/dist-packages/haystack/query.py" in __len__ > 86. self._result_count = self.query.get_count() > > File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in get_count > 619. self.run() > > File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in run > 951. results = self.backend.search(final_query, **search_kwargs) > > File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in wrapper > 33. return func(obj, query_string, *args, **kwargs) > > File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in search > 524. _source=True) > > File "/usr/local/lib/python3.7/dist-packages/elasticsearch/client/utils.py" in _wrapped > 84. return func(*args, params=params, **kwargs) > > > During handling of the above exception (search() got an unexpected keyword argument 'doc_type'), another exception occurred: I suspect this error is because indexing is still in process. I'm not certain, but I think so. > File "/usr/lib/python3/dist-packages/hyperkitty/views/search.py" in search > 90. request.GET.get('count'), > > File "/usr/lib/python3/dist-packages/django_mailman3/lib/paginator.py" in paginate > 72. objects = paginator.page(page_num) > > File "/usr/lib/python3/dist-packages/django/core/paginator.py" in page > 57. number = self.validate_number(number) > > File "/usr/lib/python3/dist-packages/django/core/paginator.py" in validate_number > 46. if number > self.num_pages: > > File "/usr/lib/python3/dist-packages/django/utils/functional.py" in __get__ > 35. res = instance.__dict__[self.name] = self.func(instance) > > File "/usr/lib/python3/dist-packages/django/core/paginator.py" in num_pages > 91. if self.count == 0 and not self.allow_empty_first_page: > > File "/usr/lib/python3/dist-packages/django/utils/functional.py" in __get__ > 35. res = instance.__dict__[self.name] = self.func(instance) > > File "/usr/lib/python3/dist-packages/django/core/paginator.py" in count > 84. return len(self.object_list) > > File "/usr/lib/python3/dist-packages/haystack/query.py" in __len__ > 86. self._result_count = self.query.get_count() > > File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in get_count > 619. self.run() > > File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in run > 951. results = self.backend.search(final_query, **search_kwargs) > > File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in wrapper > 33. return func(obj, query_string, *args, **kwargs) > > File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in search > 524. _source=True) > > File "/usr/local/lib/python3.7/dist-packages/elasticsearch/client/utils.py" in _wrapped > 84. return func(*args, params=params, **kwargs) > > > During handling of the above exception (search() got an unexpected keyword argument 'doc_type'), another exception occurred: > > > > File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner > 41. response = get_response(request) > > File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response > 187. response = self.process_exception_by_middleware(e, request) > > File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response > 185. response = wrapped_callback(request, *callback_args, **callback_kwargs) > > File "/usr/lib/python3/dist-packages/hyperkitty/views/search.py" in search > 100. if not isinstance(e, search_exception): > > Exception Type: UnboundLocalError at /hyperkitty/search > Exception Value: local variable 'search_exception' referenced before assignment This is a bug in HyperKitty. We've never encountered it, but the code at <https://gitlab.com/mailman/hyperkitty/blob/master/hyperkitty/views/search.p…> only considers the whoosh and xapian backends and needs to be updated for elasticsearch. I've reported that at <https://gitlab.com/mailman/hyperkitty/issues/263>. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years, 10 months

[MM3-users] Re: Mailman rewrite the Cc header incorrectly

by Alan So

Mark Sapiro wrote: > I haven't looked at the code to see exactly what's responsible here. I > do note that adding double quotes around a display name is not only > allowed, but is required if the display name contains "specials". So, it looks like that Outlook still consider the part (XYZ) as comment during folding even though it is within the double quotes. It is folded between DEF and (XYZ) quite often in our cases if there are four or more than four email addresses in the Cc header. Since we normally have full name in the format of "Firstname Lastname (Dept)" in our organization, we hit the issue quite easily if users send to a list with more than four addresses in the Cc header.

5 years, 3 months

[MM3-users] Re: Postorius upgrade issue

by Joel Lord

So I tried to do this after upgrading, fortunately only from 1 point release back, and I get "ModuleNotFoundError: No module named 'paintstore'" from the collectstatic. Haven't tried to get past that point, and haven't really had the time to chase this down myself. It seems like that's referring to a requirement from several versions back, but this venv was built clean only 2 months ago so it strikes me as odd. On 2/8/2021 6:18 PM, Mark Sapiro wrote: > I neglected to mention `migrate`. All 4 Django management commands, `collectstatic`, > `compress`, `compilemessages` and `migrate` should be run after any upgrade. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- Joel Lord

4 years, 7 months

[MM3-users] Re: Disabled by bounce

by Danil Smirnov

Hi Mark On Wed, Feb 1, 2023 at 9:28 PM Mark Sapiro <mark(a)msapiro.net> wrote: > Yes and no. If 'Bounce info stale after' is set to 3d and no bounces for > a user are received in a 3 day period, the user's bounce score is not > immediately reset, but it will be reset (to 1) the next time a bounce is > received for the user. I.e., it is effectively the same thing unless you > are looking at the user's bounce score which doesn't change until the > next bounce. > Isn't this a bug? We are now showing users' bounce scores on the Members page and this would be misleading. Is it difficult from internals perspective to have the bounce score reset immediately? BR, Danil Smirnov

2 years, 7 months

[MM3-users] Re: Strange password problem

by Eggert Ehmke

Am Freitag, 22. September 2023, 19:34:41 CEST schrieb Mark Sapiro: > Can you create another superuser with the Django admin createsuperuser > command and does that user's password stick? This got me on the right track. My mailman installation hosts several mailing lists that are assigned to different domains. Somehow I had created a superuser account 'admin' for each domain with different passwords. Because the browser stores the password for each domain separately, each time I log in via a different domain, I got the wrong password. So for now I use the same password for all logins, and that works. But is that a good solution? The other option would be to assign multiple superusers for each domain, but they would still allow to log in via some other domain. Thank you for your help!

1 year, 11 months

[MM3-users] Re: Chinese characters in list name shows question mark (?)

by Franklin Weng

Hi Mark, Thanks for your reply. Mark Sapiro 於 2019/10/24 上午6:10 寫道: > >> Is there any other setting I should look at? > > The list's preferred_language setting (Postorius -> Settings -> List > Identity -> Preferred Language. The list welcome message is encoded in > Mailman's character set for the list's preferred_language. If this is > English (USA), the character set is us-ascii. If it is set to Chinese > (Taiwan), the character set is utf-8, and it should work. However in my settings -> List Identity page I didn't see Preferred Language option. https://nextcloud.slat.org/index.php/s/45tGaE7ALgAFQFK What may be possible wrong that I couldn't see Preferred Language settings in the web interface? Thanks, Franklin

5 years, 10 months

[MM3-users] Re: High memory usage

by Philip Colmer

Thank you for the replies from Mark and Stephen. One of the reasons why Prasanth & I are concerned with this specific system is because the first time memory usage crept up, it got to the point where oomkiller stepped in and, as is usual for that particular mechanism, one of the processes it killed was sshd, thus removing access to the system :(. I know that oomkiller can be tuned to stop it killing processes like sshd but it will, nevertheless, start killing *some* processes if it sees an out of memory situation. So, as Prasanth said, we doubled the memory available and stuck an alarm on it to trigger when usage exceeded 85% for 5 minutes. I've saved a screenshot of the monitored memory usage over the last couple of weeks: https://1drv.ms/u/s!Aq7dn8GDGLdCodsU-pybJBdckxWU1A?e=BJY1UA (It is stored on OneDrive in case you don't recognise the URL format) We changed the server on 4th February to the larger memory size. On 11th Feb, it triggered the alarm and we rebooted the server. Memory usage is starting to climb again, albeit not at the pace that it was previously. What is particularly curious for us is that we have four Mailman 3 servers. Two of them are single domain and two are hosting two domains. This server is one of the servers hosting two domains, but it is also the ONLY one of the four servers exhibiting this memory pattern. By comparison, the memory usage for the other server hosting two domains looks like this: https://1drv.ms/u/s!Aq7dn8GDGLdCodsVyEufkeQqit_cGA?e=ifrJHs The biggest difference we are aware of with the "problematic" server is that some of the list archives are quite large by comparison with our other servers. The largest is 15K messages, then 2K then 1K. The other three servers have lists with relatively small archives by comparison (double or triple digits at worst). Finally, and just FYI, we're using PostgreSQL for the database and Xapian for the indexing engine. Regards Philip On Tue, 15 Feb 2022 at 16:36, Mark Sapiro <mark(a)msapiro.net> wrote: > On 2/15/22 08:11, Stephen Daniel wrote: > > TL;DR: quite likely your system has plenty of memory and is operating > > normally. > ... > > On Tue, Feb 15, 2022 at 5:28 AM Prasanth Nair <prasanth.nair(a)linaro.org> > > wrote: > > > >> Hi, > >> > >> We are having an issue with high memory usage on our Mailman3 server. > >> Initially, our server had 16GB RAM and the memory usage went up from 5% > to > >> 92% within a week. So, we increased the memory capacity to 32GB RAM and > we > >> can see that the memory usage is going up in the same pattern. > >> > >> Any idea why it is happening? or any suggestions? > > > Just for reference, mail.python.org supports 231 MM 2 lists and 182 MM 3 > lists with 16 GB of ram and no swap. > > The server that supports this list and also the www.list.org web site > has 4 GB of ram and no swap. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

3 years, 6 months

[MM3-users] Re: Hyperkitty - Non-ascii sender address

by Thomas G

Hi, https://gitlab.com/mailman/hyperkitty/blob/master/hyperkitty/lib/incoming.p… I changed "sender" for this; # Sender try: from_str = header_to_unicode(message['From']) from_name, from_email = parseaddr(from_str) from_name = unidecode.unidecode(from_name).strip() except (UnicodeDecodeError, UnicodeEncodeError): raise (ValueError("Non-ascii sender address", message)) try: sender_address = from_email.encode('ascii').decode("ascii").strip() except (UnicodeDecodeError, UnicodeEncodeError): if from_name: sender_address = re.sub("[^a-z0-9]", "", from_name.lower()) if not sender_address: sender_address = "unknown(a)unknown.com" else: sender_address = "unknown(a)unknown.com" print("Non-ascii sender addre -- Sender address replaced by ", sender_address) I got this output (example): |Non-ascii sender address -- Sender address replaced by stephaneblipblopstlablipblopgmailcom It's not really perfect, I'm working to do something better, but basically it's works. I think that there is a issue in the original code: Because : "" raise ValueError("Non-ascii sender address", message) "" The second part will be never call: if not sender_address: if from_name: sender_address = re.sub("[^a-z0-9]", "", from_name.lower()) if not sender_address: sender_address = "unknown" sender_address = "{}(a)example.com".format(sender_address) else: sender_address = "unknown(a)example.com" If you bypass : "" raise ValueError("Non-ascii sender address", message) "", by a simple print(), you will get an error due to an unassigned variable (sender_address). Le mar. 30 oct. 2018 à 04:10, Mark Sapiro <mark(a)msapiro.net> a écrit : > On 10/28/18 12:03 PM, Guiseppin Thomas wrote: > > Hi, > > > > I imported some mbox archives in Hyperkitty. > > > > I have lot issues like: > > > > /\Failed adding message <5b0900ab.1c69fb81.70ab.2f87(a)mx.google.com>: > > ('Non-ascii sender address', <email.message.EmailMessage object at > > 0x7f05f1606048>)// > > //Non-ascii sender address from "Stéphane blipblop <stlablipblop at > > gmail.com>" about [HATLAB][It] Re: A débarrasser ! / > > > > It's probably because "Stéphane". > > > The 'Non-ascii sender address' message comes from > > > try: > > from_str = header_to_unicode(message['From']) > > from_name, from_email = parseaddr(from_str) > > from_name = from_name.strip() > > sender_address = > from_email.encode('ascii').decode("ascii").strip() > > except (UnicodeDecodeError, UnicodeEncodeError): > > raise ValueError("Non-ascii sender address", message) > > Possibly the exception is thrown in parseaddr(), but much more likely, > the email address, not the display name contains non-ascii. > > > Some advise to fix it ? > > > All the other messages should have been imported. If you can find and > fix the 'bad' ones in the mbox, you can make a new mbox with just those > and import that. You could also import the entire mbox with fixed > messages. The already added ones won't be re-imported because of > duplicate Message-ID:, but it's extra processing. > > If the actual bad message From: header is > > From: Stéphane blipblop <stlablipblop(a)gmail.com> > > or > > From: Stéphane blipblop <stlablipblop at gmail.com> > > That shouldn't cause the error, so I'g guessing that 'blipblop' is not > the actual value and the actual value in <stlablipblop at gmail.com> has > non-ascii. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

6 years, 10 months

[MM3-users] Re: remove non members from a list

by Guillermo Hernandez (Oldno7)

On 6/8/21 19:16, Mark Sapiro wrote: > On 8/6/21 9:48 AM, Guillermo Hernandez (Oldno7) via Mailman-users wrote: >> >> It's not functioning. It says: >> >> "AttributeError: 'MemberRoster' object has no attribute 'nonmembers'" > > Sorry, that was a typo on my part. It should be > > >>> for nonmember in m.nonmembers.members: > ... nonmember.unsubscribe() > ... > >>> commit() > > Thanks. Now it's no showing errors (and nothing at all) but apparently it does nothing. The non mebers of the list (the few that remains: I've been deleting one at a time from the web) aparently still are there.. Knowing the correct procedure will ease me to try tomorrow with a clean head. Thanks a lot.

4 years, 1 month

[MM3-users] Re: Mailman3 reinstall fail, need some help

by Yvan ♞

Ok thanks, I'll try on debian forum as I already unsuccessfully tried on yunohost since 6 months. Le 14/03/2022 à 16:23, Mark Sapiro a écrit : > On 3/14/22 08:12, Yvan ♞ wrote: >> Maybe an idea of my problem as I have pgadmin app installed, it could >> be in conflict to create a postgresql db? > > Your original dbug logs seem to show no issue with postgresql. > > I don't think people on this list can be much help for installing the > Debian package on YunoHost. As I said earlier, I would go first to > YunoHost for help and then to Debian if necessary. > > If you want help from us, I recommend installing as documented at > https://docs.mailman3.org/en/latest/install/virtualenv.html >

3 years, 5 months

Search results for query "sapiro"