Mailman3 on AlmaLinux 9 - Python3.9 errors
Hi, I'm trying to configure mailman3 on my AlmaLinux 9 server and when checking the mailman3 status, I get the following:
# mailman3 status Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 227, in _catch_revision_errors yield File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 439, in _upgrade_revs for script in reversed(list(revs)) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 793, in iterate_revisions revisions, heads = fn( File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 1420, in _collect_upgrade_revisions current_revisions = self.get_revisions(lower) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 525, in get_revisions return sum([self.get_revisions(id_elem) for id_elem in id_], ()) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 525, in <listcomp> return sum([self.get_revisions(id_elem) for id_elem in id_], ()) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 549, in get_revisions return tuple( File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 550, in <genexpr> self._revision_for_ident(rev_id, branch_label) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 619, in _revision_for_ident raise ResolutionError( alembic.script.revision.ResolutionError: No such revision or branch '2156fc3f6f7d'
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/libexec/mailman3/mailman", line 8, in <module> sys.exit(main()) File "/usr/lib/python3.9/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3.9/site-packages/click/core.py", line 1052, in main with self.make_context(prog_name, args, **extra) as ctx: File "/usr/lib/python3.9/site-packages/click/core.py", line 914, in make_context self.parse_args(ctx, args) File "/usr/lib/python3.9/site-packages/click/core.py", line 1615, in parse_args rest = super().parse_args(ctx, args) File "/usr/lib/python3.9/site-packages/click/core.py", line 1370, in parse_args value, args = param.handle_parse_result(ctx, opts, args) File "/usr/lib/python3.9/site-packages/click/core.py", line 2347, in handle_parse_result value = self.process_value(ctx, value) File "/usr/lib/python3.9/site-packages/click/core.py", line 2309, in process_value value = self.callback(ctx, self, value) File "/usr/lib/python3.9/site-packages/mailman/bin/mailman.py", line 95, in initialize_config initialize(value) File "/usr/lib/python3.9/site-packages/mailman/core/initialize.py", line 229, in initialize initialize_2(propagate_logs=propagate_logs) File "/usr/lib/python3.9/site-packages/mailman/core/initialize.py", line 187, in initialize_2 config.db = getUtility(IDatabaseFactory, utility_name).create() File "/usr/lib/python3.9/site-packages/mailman/database/factory.py", line 58, in create SchemaManager(database).setup_database() File "/usr/lib/python3.9/site-packages/mailman/database/factory.py", line 109, in setup_database alembic.command.upgrade(alembic_cfg, 'head') File "/usr/lib/python3.9/site-packages/alembic/command.py", line 320, in upgrade script.run_env() File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 563, in run_env util.load_python_file(self.dir, "env.py") File "/usr/lib/python3.9/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file module = load_module_py(module_id, path) File "/usr/lib/python3.9/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py spec.loader.exec_module(module) # type: ignore File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/usr/lib/python3.9/site-packages/mailman/database/alembic/env.py", line 77, in <module> run_migrations_online() File "/usr/lib/python3.9/site-packages/mailman/database/alembic/env.py", line 71, in run_migrations_online context.run_migrations() File "<string>", line 8, in run_migrations File "/usr/lib/python3.9/site-packages/alembic/runtime/environment.py", line 851, in run_migrations self.get_context().run_migrations(**kw) File "/usr/lib/python3.9/site-packages/alembic/runtime/migration.py", line 608, in run_migrations for step in self._migrations_fn(heads, self): File "/usr/lib/python3.9/site-packages/alembic/command.py", line 309, in upgrade return script._upgrade_revs(revision, rev) File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 435, in _upgrade_revs return [ File "/usr/lib64/python3.9/contextlib.py", line 137, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 259, in _catch_revision_errors raise util.CommandError(resolution) from re alembic.util.exc.CommandError: Can't locate revision identified by '2156fc3f6f7d'
Any ideas why?
On 7/1/24 12:34, willie.castillo@caemilusa.com wrote:
Hi, I'm trying to configure mailman3 on my AlmaLinux 9 server and when checking the mailman3 status, I get the following: ... File "<string>", line 8, in run_migrations File "/usr/lib/python3.9/site-packages/alembic/runtime/environment.py", line 851, in run_migrations self.get_context().run_migrations(**kw) File "/usr/lib/python3.9/site-packages/alembic/runtime/migration.py", line 608, in run_migrations for step in self._migrations_fn(heads, self): File "/usr/lib/python3.9/site-packages/alembic/command.py", line 309, in upgrade return script._upgrade_revs(revision, rev) File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 435, in _upgrade_revs return [ File "/usr/lib64/python3.9/contextlib.py", line 137, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 259, in _catch_revision_errors raise util.CommandError(resolution) from re alembic.util.exc.CommandError: Can't locate revision identified by '2156fc3f6f7d'
Any ideas why?
It looks like your mailman/database/alembic/versions/ directory is missing 2156fc3f6f7d_add_dmarc_addresses.py. What is your mailman version? This migration was added in Mailman core 3.3.9.
Possibly your Mailman version is < 3.3.9 but the alembic_version table in your mailman database points to an alembic version not in your mailman core version.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Not sure if there is a configuration issue here, but I'm getting this when trying to start mailman3.service... I'm running mailman3 on an AlmaLinux 9.4 × mailman3.service - GNU Mailing List Manager Loaded: loaded (/usr/lib/systemd/system/mailman3.service; enabled; preset: disabled) Active: failed (Result: exit-code) since Wed 2024-07-17 11:26:29 EDT; 29min ago Process: 1099 ExecStart=/usr/bin/mailman3 start --force (code=exited, status=1/FAILURE) CPU: 962ms
Jul 17 11:26:29 tpalxinflstp01.caemilusa.com mailman3[1119]: File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 435, in _upgrade_revs Jul 17 11:26:29 tpalxinflstp01.caemilusa.com mailman3[1119]: return [ Jul 17 11:26:29 tpalxinflstp01.caemilusa.com mailman3[1119]: File "/usr/lib64/python3.9/contextlib.py", line 137, in __exit__ Jul 17 11:26:29 tpalxinflstp01.caemilusa.com mailman3[1119]: self.gen.throw(typ, value, traceback) Jul 17 11:26:29 tpalxinflstp01.caemilusa.com mailman3[1119]: File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 259, in _catch_revision_erro> Jul 17 11:26:29 tpalxinflstp01.caemilusa.com mailman3[1119]: raise util.CommandError(resolution) from re Jul 17 11:26:29 tpalxinflstp01.caemilusa.com mailman3[1119]: alembic.util.exc.CommandError: Can't locate revision identified by '2156fc3f6f7d' Jul 17 11:26:29 tpalxinflstp01.caemilusa.com systemd[1]: mailman3.service: Control process exited, code=exited, status=1/FAILURE Jul 17 11:26:29 tpalxinflstp01.caemilusa.com systemd[1]: mailman3.service: Failed with result 'exit-code'. Jul 17 11:26:29 tpalxinflstp01.caemilusa.com systemd[1]: Failed to start GNU Mailing List Manager.
I see the file /usr/local/lib/python3.9/site-packages/mailman/database/alembic/versions/2156fc3f6f7d_add_dmarc_addresses.py
On 7/17/24 9:49 AM, willie.castillo@caemilusa.com wrote:
I see the file /usr/local/lib/python3.9/site-packages/mailman/database/alembic/versions/2156fc3f6f7d_add_dmarc_addresses.py
Yes, this is strange. What does /usr/bin/mailman3 info
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Not sure what's going on here. This is the output of the "mailman3 info". What am I doing wrong?
$ /usr/bin/mailman3 info Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 227, in _catch_revision_errors yield File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 439, in _upgrade_revs for script in reversed(list(revs)) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 793, in iterate_revisions revisions, heads = fn( File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 1420, in _collect_upgrade_revisions current_revisions = self.get_revisions(lower) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 525, in get_revisions return sum([self.get_revisions(id_elem) for id_elem in id_], ()) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 525, in <listcomp> return sum([self.get_revisions(id_elem) for id_elem in id_], ()) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 549, in get_revisions return tuple( File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 550, in <genexpr> self._revision_for_ident(rev_id, branch_label) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 619, in _revision_for_ident raise ResolutionError( alembic.script.revision.ResolutionError: No such revision or branch '2156fc3f6f7d'
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/libexec/mailman3/mailman", line 8, in <module> sys.exit(main()) File "/usr/lib/python3.9/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3.9/site-packages/click/core.py", line 1052, in main with self.make_context(prog_name, args, **extra) as ctx: File "/usr/lib/python3.9/site-packages/click/core.py", line 914, in make_context self.parse_args(ctx, args) File "/usr/lib/python3.9/site-packages/click/core.py", line 1615, in parse_args rest = super().parse_args(ctx, args) File "/usr/lib/python3.9/site-packages/click/core.py", line 1370, in parse_args value, args = param.handle_parse_result(ctx, opts, args) File "/usr/lib/python3.9/site-packages/click/core.py", line 2347, in handle_parse_result value = self.process_value(ctx, value) File "/usr/lib/python3.9/site-packages/click/core.py", line 2309, in process_value value = self.callback(ctx, self, value) File "/usr/lib/python3.9/site-packages/mailman/bin/mailman.py", line 95, in initialize_config initialize(value) File "/usr/lib/python3.9/site-packages/mailman/core/initialize.py", line 229, in initialize initialize_2(propagate_logs=propagate_logs) File "/usr/lib/python3.9/site-packages/mailman/core/initialize.py", line 187, in initialize_2 config.db = getUtility(IDatabaseFactory, utility_name).create() File "/usr/lib/python3.9/site-packages/mailman/database/factory.py", line 58, in create SchemaManager(database).setup_database() File "/usr/lib/python3.9/site-packages/mailman/database/factory.py", line 109, in setup_database alembic.command.upgrade(alembic_cfg, 'head') File "/usr/lib/python3.9/site-packages/alembic/command.py", line 320, in upgrade script.run_env() File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 563, in run_env util.load_python_file(self.dir, "env.py") File "/usr/lib/python3.9/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file module = load_module_py(module_id, path) File "/usr/lib/python3.9/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py spec.loader.exec_module(module) # type: ignore File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/usr/lib/python3.9/site-packages/mailman/database/alembic/env.py", line 77, in <module> run_migrations_online() File "/usr/lib/python3.9/site-packages/mailman/database/alembic/env.py", line 71, in run_migrations_online context.run_migrations() File "<string>", line 8, in run_migrations File "/usr/lib/python3.9/site-packages/alembic/runtime/environment.py", line 851, in run_migrations self.get_context().run_migrations(**kw) File "/usr/lib/python3.9/site-packages/alembic/runtime/migration.py", line 608, in run_migrations for step in self._migrations_fn(heads, self): File "/usr/lib/python3.9/site-packages/alembic/command.py", line 309, in upgrade return script._upgrade_revs(revision, rev) File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 435, in _upgrade_revs return [ File "/usr/lib64/python3.9/contextlib.py", line 137, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 259, in _catch_revision_errors raise util.CommandError(resolution) from re alembic.util.exc.CommandError: Can't locate revision identified by '2156fc3f6f7d'
On 7/18/24 7:58 AM, willie.castillo@caemilusa.com wrote:
Not sure what's going on here. This is the output of the "mailman3 info". What am I doing wrong?
$ /usr/bin/mailman3 info Traceback (most recent call last): ... alembic.script.revision.ResolutionError: No such revision or branch '2156fc3f6f7d'
The mailman3 command is trying to run all the alembic migrations as
part of initialization and is failing with this error. Thus, it never
gets to the point of actually doing what you ask.
What is the version_num in the alembic_version table in the database? I.e.,
SELECT version_num FROM alembic_version;
and what does
ls /usr/lib/python3.9/site-packages/mailman/database/alembic/versions/
and
grep -r 2156fc3f6f7d
/usr/lib/python3.9/site-packages/mailman/database/alembic/versions/
give?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark first let me thank you for your assistance with this. I don't believe I mentioned that this is a fresh/new installation I'm performing on an AlmaLinux 9.4 VM. I have searched everywhere for a solid Mailman installation on AlmaLinux document/process but, I haven't had any luck. I tried following documentation on this site and I only found this https://docs.mailman3.org/en/latest/install/distro.html.
I have been trying to put all necessary pieces together (just like a puzzle) but, I haven't been successful in getting to the finish line.
Do you happen to know where I can find step-by-step Mailman3 installation for AlmaLinux
I have nginx, postfix running but, cannot get mailman3 running.
● nginx.service - The nginx HTTP and reverse proxy server Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; preset: disabled) Active: active (running) since Thu 2024-07-18 13:54:17 EDT; 17min ago Process: 1107 ExecStartPre=/usr/bin/rm -f /run/nginx.pid (code=exited, status=0/SUCCESS) Process: 1110 ExecStartPre=/usr/sbin/nginx -t (code=exited, status=0/SUCCESS) Process: 1183 ExecStart=/usr/sbin/nginx (code=exited, status=0/SUCCESS) Main PID: 1185 (nginx) Tasks: 5 (limit: 48930) Memory: 6.9M CPU: 377ms CGroup: /system.slice/nginx.service ├─1185 "nginx: master process /usr/sbin/nginx" ├─1186 "nginx: worker process" ├─1187 "nginx: worker process" ├─1188 "nginx: worker process" └─1189 "nginx: worker process"
Jul 18 13:54:17 tpalxinflstp01.caemilusa.com systemd[1]: Starting The nginx HTTP and reverse proxy server... Jul 18 13:54:17 tpalxinflstp01.caemilusa.com nginx[1110]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok Jul 18 13:54:17 tpalxinflstp01.caemilusa.com nginx[1110]: nginx: configuration file /etc/nginx/nginx.conf test is successful Jul 18 13:54:17 tpalxinflstp01.caemilusa.com systemd[1]: Started The nginx HTTP and reverse proxy server.
● postfix.service - Postfix Mail Transport Agent Loaded: loaded (/usr/lib/systemd/system/postfix.service; enabled; preset: disabled) Active: active (running) since Thu 2024-07-18 13:54:18 EDT; 17min ago Process: 1094 ExecStartPre=/usr/sbin/restorecon -R /var/spool/postfix/pid (code=exited, status=0/SUCCESS) Process: 1103 ExecStartPre=/usr/libexec/postfix/aliasesdb (code=exited, status=0/SUCCESS) Process: 1153 ExecStartPre=/usr/libexec/postfix/chroot-update (code=exited, status=0/SUCCESS) Process: 1160 ExecStart=/usr/sbin/postfix start (code=exited, status=0/SUCCESS) Main PID: 1439 (master) Tasks: 3 (limit: 48930) Memory: 9.1M CPU: 238ms CGroup: /system.slice/postfix.service ├─1439 /usr/libexec/postfix/master -w ├─1471 pickup -l -t unix -u └─1472 qmgr -l -t unix -u
Jul 18 13:54:17 tpalxinflstp01.caemilusa.com systemd[1]: Starting Postfix Mail Transport Agent... Jul 18 13:54:18 tpalxinflstp01.caemilusa.com postfix/postfix-script[1427]: starting the Postfix mail system Jul 18 13:54:18 tpalxinflstp01.caemilusa.com postfix/master[1439]: daemon started -- version 3.5.9, configuration /etc/postfix Jul 18 13:54:18 tpalxinflstp01.caemilusa.com systemd[1]: Started Postfix Mail Transport Agent.
× mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; preset: disabled) Active: failed (Result: exit-code) since Thu 2024-07-18 13:54:20 EDT; 17min ago Process: 1477 ExecStart=/usr/bin/mailman3 start --force (code=exited, status=1/FAILURE) CPU: 642ms
Jul 18 13:54:19 tpalxinflstp01.caemilusa.com mailman3[1496]: File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 435, in _upgrade_revs Jul 18 13:54:19 tpalxinflstp01.caemilusa.com mailman3[1496]: return [ Jul 18 13:54:19 tpalxinflstp01.caemilusa.com mailman3[1496]: File "/usr/lib64/python3.9/contextlib.py", line 137, in __exit__ Jul 18 13:54:19 tpalxinflstp01.caemilusa.com mailman3[1496]: self.gen.throw(typ, value, traceback) Jul 18 13:54:19 tpalxinflstp01.caemilusa.com mailman3[1496]: File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 259, in _catch_revision_errors Jul 18 13:54:19 tpalxinflstp01.caemilusa.com mailman3[1496]: raise util.CommandError(resolution) from re Jul 18 13:54:19 tpalxinflstp01.caemilusa.com mailman3[1496]: alembic.util.exc.CommandError: Can't locate revision identified by '2156fc3f6f7d' Jul 18 13:54:20 tpalxinflstp01.caemilusa.com systemd[1]: mailman3.service: Control process exited, code=exited, status=1/FAILURE Jul 18 13:54:20 tpalxinflstp01.caemilusa.com systemd[1]: mailman3.service: Failed with result 'exit-code'. Jul 18 13:54:20 tpalxinflstp01.caemilusa.com systemd[1]: Failed to start GNU Mailing List Manager.
On 7/18/24 11:12 AM, willie.castillo@caemilusa.com wrote:
Mark first let me thank you for your assistance with this. I don't believe I mentioned that this is a fresh/new installation I'm performing on an AlmaLinux 9.4 VM. I have searched everywhere for a solid Mailman installation on AlmaLinux document/process but, I haven't had any luck. I tried following documentation on this site and I only found this https://docs.mailman3.org/en/latest/install/distro.html.
Did you see my post at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
We recommend installing Pypi packages in a virtualenv. Doc is at https://docs.mailman3.org/en/latest/install/virtualenv.html
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I saw them. OK, I must be missing several steps. I'll have to start from scratch. Let me carefully follow the link you send me, it should work for Almalinux 9.4 right?
We recommend installing Pypi packages in a virtualenv. Doc is at https://urldefense.us/v3/__https://docs.mailman3.org/en/latest/install/virtu...
[cid:92ffd146-5b65-4e46-a2f8-021ea40371e9]
Willie Castillo
Linux Systems Administrator, IT Department - Defense & Security
CAE USA
T. +1-813-885-7481 E. Willie.Castillo@caemilusa.com
A. 4908 Tampa West Blvd, Tampa, Fl. 33634
cae.com
CONFIDENTIALITY NOTICE
This e-mail message is intended only for the above named recipient(s) and may contain information that is confidential. If you have received this message in error or are not the named recipient(s), please immediately notify the sender, delete this e-mail message without making a copy and do not disclose or relay this e-mail message to anyone.
From: Mark Sapiro <mark@msapiro.net> Sent: Thursday, July 18, 2024 2:55 PM To: mailman-users@mailman3.org <mailman-users@mailman3.org> Subject: [NON-INTERNAL] [MM3-users] Re: Mailman3 on AlmaLinux 9 - Python3.9 errors
[You don't often get email from mark@msapiro.net. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On 7/18/24 11:12 AM, willie.castillo@caemilusa.com wrote:
Mark first let me thank you for your assistance with this. I don't believe I mentioned that this is a fresh/new installation I'm performing on an AlmaLinux 9.4 VM. I have searched everywhere for a solid Mailman installation on AlmaLinux document/process but, I haven't had any luck. I tried following documentation on this site and I only found this https://urldefense.us/v3/__https://docs.mailman3.org/en/latest/install/distr... .
Did you see my post at https://urldefense.us/v3/__https://lists.mailman3.org/archives/list/mailman-...
We recommend installing Pypi packages in a virtualenv. Doc is at https://urldefense.us/v3/__https://docs.mailman3.org/en/latest/install/virtu...
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://urldefense.us/v3/__https://lists.mailman3.org/mailman3/lists/mailman... Archived at: https://urldefense.us/v3/__https://lists.mailman3.org/archives/list/mailman-...
This message sent to willie.castillo@caemilusa.com
Mailman's content filtering has removed the following MIME parts from this message.
Content-Type: image/png Name: Outlook-2rvyqd2h.png
Replaced multipart/alternative part with first alternative.
On Thu, Jul 18, 2024 at 10:46 PM Willie Castillo < Willie.Castillo@caemilusa.com> wrote:
I saw them. OK, I must be missing several steps. I'll have to start from scratch. Let me carefully follow the link you send me, it should work for Almalinux 9.4 right?
<odhiambo@gmail.com> For ALL *nix :)
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
participants (4)
-
Mark Sapiro -
Odhiambo Washington -
Willie Castillo -
willie.castillo＠caemilusa.com