Dennis Gorelik

1) Today (2020-05-04) my email client (The Bat 9.1.18 Professional) - lost ability to receive or send emails from our corporate @postjobfree.com email addresses [that G-suit handles for us].

The Bat started to return "Server reports error. The response is: 5.7.0":
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
!2020-05-04, 06:18:43: SEND - Server reports error. The response is: 5.7.0 https://support.google.com/mail/?p=WantAuthError h188sm3660946qke.82 - gsmtp
2020-05-04, 06:18:43: SEND - connection finished - 0 message(s) sent
2020-05-04, 06:18:43: SEND - Some messages were not sent - check the log for details
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(Times are in EST)

2) The reason for this error is that Google [temporarily] stopped supporting "Less Secure Apps" in G-Suite (which we use to manage postjobfree.com emails).

3) This instruction: https://hotter.io/docs/email-accounts/secure-app-gmail/ is now obsolete.
"Less secure app access panel" no longer exists on https://myaccount.google.com/security page.

4) Actually, less secure access to Gmail works again, but, most likely, only temporary.
=======
https://gsuiteupdates.googleblog.com/2020/03/less-secure-app-turn-off-suspended.html
Last December, we announced that we’d be turning off less secure app (LSA) access to G Suite accounts, and that you should migrate to OAuth authentication instead. The first phase of the LSA turn-down was scheduled for June 15, 2020. As many organizations deal with the impact of COVID-19 and are now focused on supporting a remote workforce, we want to minimize potential disruptions for customers unable to complete migrations in this timeframe.
As a result, we are suspending the LSA turn-off until further notice. All previously announced timeframes no longer apply.
...
Despite these timing adjustments, Google does not recommend the use of any application that does not support OAuth.
=======

5) This is the fix for this The Bat - Gmail issue:
~~~~~~~~~~~~~
https://www.ritlabs.com/en/news/6777/
To configure OAuth for existing Gmail or Mail.Ru accounts in The Bat!, users should access the "Account -> Properties -> Transport -> Receive mail -> Authentication" menu, enable "OAUTH (using Web Browser)" and save changes. The Bat! will remove the password from the account and apply the modern authentication mechanism to send and receive messages.
~~~~~~~~~~~~~

Click "Authentication":


Switch from "Regular" to "OAUTH (using Web Browser)":


6) The Bat works again.

7) The next step - is to switch our PostJobFreeService C# code that works with Gmail over [secure] Smtp/Pop3 to OAuth.

I accidentally clicked on "Update Date" on one of my older posting.
How do I revert that posting date to the original "Posted Date"?

service@paypal.com emailed to me and asked to verify one of our developer's identity ... by faxing documents to:
=========
(402) 573-5732 (Attn: PayPal Compliance Department).
=========

It looks like PayPal jumped into a time machine and went back to the previous century.


service@paypal.com did NOT list any other options to upload documents.
The email ended with: "Please do not reply to this email. This mailbox is not monitored and you will not receive a response."

I replied anyway.

service@paypal.com sent this auto-response:
~~~~~~~~~~~~~~
Thanks for contacting us.

We want to help you, but we can't answer emails sent to this unmonitored
mailbox.
~~~~~~~~~~~~~~


My resolution -- delete that developer account from the PayPal account.
Let's see if it would help.

Today (and few days ago) I talked with a founder of botson.ai - Dan.
Dan told me how amazing their tech team is in chatbots and jobs matching technology.

When I asked Dan about specific details about what made their job matching algorithms so amazing -- Dan told me that integration of their javascript into our job board is super-easy.

That javascript would tag job seekers who visit our job board with botson.ai cookie.
If job seeker has an open Facebook session - then botson.ai would sign job seeker up for jobs chatbot on Facebook messenger.
Then the chatbot would send jobs that it thinks are relevant to what job seeker is searching for.

I wanted to try that amazing technology before integrating it into our job board and asked Dan to give me a link to signup.
Dan gave me a link, but it did not work. Dan gave me another link. The second link did not work either. The third link worked and I received Facebook message from "Maya - Recruiter".
"Maya" was moderate and only messaged me when I messaged her.
However job matches were not relevant.
I called myself "Monster Botsonai" in a signup form (on the affiliate web site that Dan gave me links me to), so Maya initially called me "Monster", but then corrected itself based on my Facebook profile.

This is my chat with "Maya - Recruiter":
==================
[Maya]
Hi Monster 😊
Following your request for jobs near 32081 here is a selection just for you

[Facebook]
You agreed to have Maya - Recruiter send you updates. Replying will allow Maya - Recruiter to see info you've made public.

[Maya]
Warehouse Associate
We're Hiring Warehouse Associates!. Pay Rate Picking and packing Loading and un...
More Details
Apply Now

[Dennis]
View More Jobs

[Maya]
Pool Technician II
Position Summary:This position is primarily responsible for the daily maintenanc...
More Details
Apply Now

[Dennis]
I look for different jobs

[Maya]
Experienced Auto Hauler
We are a new and growing company that is actively seeking experienced auto haule...
More Details
Apply Now

[Dennis]
[Clicked on "Caregiver jobs"]

[Maya]
Caregiver / Resident Assistant / Personal Care Assistant / CNA
Join a Family, Serve with the Best - Benton House at OakleafAre you a CNA lookin...
More Details
Apply Now

[Dennis]
Can you send me different jobs?

[Maya]
Order Puller - Warehouse
Join the Remedy Family ~ Build Your Career The Remedy Staffing Jacksonville Team...
More Details
Apply Now

[Dennis]
What about C# sql jobs?

[Maya]
Local Landscapers Wanted!
Over 25 million homeowners have trusted HomeAdvisor 1800Contractor to help the...
More Details
Apply Now

[Dennis]
SQL and C#

[Maya]
Hi 😊 I'm an automated HR service and I can only assist you to find more job offers.

I’ll do my best to help you with your search. What job position you are looking for?

[Dennis]
c# sql

[Maya]
Hi 😊 I'm an automated HR service and I can only assist you to find more job offers.

I’ll do my best to help you with your search. What job position you are looking for?

[Dennis]
Software developer

[Maya]
IT Software Developer Senior
EverBank is more than just a bank; it's the evolution of an idea. Traditional fi...
More Details
Apply Now

[Dennis]
C#

[Maya]
Hi 😊 I'm an automated HR service and I can only assist you to find more job offers.

I’ll do my best to help you with your search. What job position you are looking for?

Dennis, was this helpful for you?
Do you want to see more job openings? Just tell me what kind of job position you are looking for 😊

[Dennis]
oncology nurse

[Maya]
Hi 😊 I'm an automated HR service and I can only assist you to find more job offers.

I’ll do my best to help you with your search. What job position you are looking for?

[Dennis]
nurse

[Maya]
Registered Nurse (RN)PRN - Endoscopy
Job ID: 198390Registered Nurse (RN)PRN - EndoscopyJacksonville, FloridaRegular/R...
More Details
Apply Now
==================

So it looks like Botson.ai chatbot recognizes some large job search categories (such as "software developer" and "nurse"), but does not recognize more specific skills (C#, SQL, oncology).

My conclusion is that the intelligence behind the chatbot technology is not ready yet, but the idea to use chatbot for job search - looks interesting.

We are using Amazon SES to send over 10M emails per month.
This is cheap and reliable service, and overall I like it.
However customer support is poor.

Recently, Amazon SES team removed 2 important graphs from their SES dashboard: "Bounce rate" and "Complaints".
Apparently, Amazon SES team decided that customers would be happier to build their own reports on buggy CloudWatch instead of using already prepared report.

I complained that "development" on AWS forum, and here is the reply:
https://forums.aws.amazon.com/thread.jspa?messageID=803845#803845

The proposed solution - does NOT work.

To add an insult to the injury, AWS forum does not allow me to reply yet with a message: "Your message quota has been reached. Please try again later."

Here is my forum reply that I am NOT able to post on AWS forum yet:

===============
@AWSBrentM

1) Thank you for trying to help.
I followed your steps and this produced two graphs to me.
Both graphs have constant horizontal lines at 1.00 level.
So, pretty much, both graphs are totally useless.
https://us-west-2.console.aws.amazon.com/cloudwatch/home?region=us-west-2#dashboards:name=ForumTip

Should I use instead different "metrics" instead of "Bounce" and "Complaint"?
"Reputation.BounceRate" and "Reputation.ComplaintRate" perhaps?

2) Do you know why Amazon SES Team decided to make us (its customers) to jump through the hoops of creating these awkward custom reports instead of just keeping already existing functionality?
This exercise is time-consuming, and while I am troubleshooting cloudwatch app - I am NOT building my own app that pays the bills for all of us.

3) "Your message quota has been reached. Please try again later. " ... this is my second message (and the first message in ~12 hours).
It looks like Amazon SES Team is not eager about receiving feedback...
===============

Accidentally, few days ago I received email from SendGrid rep that suggested me it may be a time to switch away from Amazon SES to SendGrid in order to get a superior customer support.
I asked SendGrid rep what kind of support I may need, got no clear reply. But Amazon SES team seems to be happy to prove their competitors right.

See also: https://dennisgorelik.dreamwidth.org/tag/amazon+ses

ElasticSearch team defends the bloat in ElasticSearch Percolator 5.4
--------
https://github.com/elastic/elasticsearch/issues/25308
If you're not interested in ranking you can easily turn it off, by wrapping the percolate query in a constant_score query.
.....
The percolator tries to tag the queries automatically based on the containing query terms. However it can't do this for all percolator queries, because the percolator doesn't know how to extract meaningful information during indexing for all queries. This is a work in progress and will get better over time. It already has shown a significant performance improvement for cases where the percolator was able to analyze the percolator query correctly at index time.
--------

1) Funny how in order to turn off unneeded feature, application developers have to create an extra wrapper around their query.

2) "work in progress" did not stop ElasticSearch team from breaking backward compatibility and forcing their users to rewrite their legacy code in favor of "work in progress" ElasticSearch 5.4.

3) "a significant performance improvement" is not quantified, and the cases where that improvement happened -
not described.

See also: ElasticSearch Percolator Bloat - part 1

Today I learned that Spark programming language has nothing to do with Apache Spark.

That "Spark Language vs Apache Spark" confusion is even worse than "Java vs Javascript" confusion.

Our web crawler hanged. Again.
It looks like it hangs with the frequency of about 1 hang per 1 million web page downloads.
Does not timeout. Does not crash. Just hangs.
Fortunately, other threads in our PostJobFreeService keep running.
(Until AngleSharp HTML parser would crash the whole PostJobFreeService on some weird HTML page, of course.)

Unfortunately, crawler hang is not reproducible: the same page can be downloaded without any problems on the next attempt. Or just in a browser.
But once in 1M downloads something weird happens: our crawler successfully passes HTTP handshake with the remote web server (so no HTTP connection timeout), but then hangs.

For our crawler we are using standard HttpWebRequest class from .NET framework.
Should we crawl with something else?

Or is it inevitable that web crawler would hang eventually and our watchdog should simply restart corresponding thread?

Discussion in Livejournal: http://dennisgorelik.livejournal.com/124693.html

Normally, in case of invalid input Uri() code throws UriFormatException. But with really weird input Uri(baseUri, Url) overload can produce NullReferenceException:

[TestMethod]
[ExpectedException(typeof(NullReferenceException))]
public void UriFailureTest()
{
    new Uri(
        new Uri("https://jobs.web.cern.ch/content/cern-jobs-insight/what-are-we-doing-while-you%E2%80%99re-waiting"),
        "https:/jobs.web.cern.ch/content/cern-jobs-insight/what-are-we-doing-while-you%E2%80%99re-waiting");
}

We tried to run PowerShell remotely (in order to automate build deployment).
We managed to make it work on developers' machines, but on production server it just refuses to work:
----------------------------------------------------------------
C:\Windows\system32>powershell
Windows PowerShell

PS C:\Windows\system32> Enable-PSRemoting -SkipNetworkProfileCheck -Force

WinRM is already set up to receive requests on this computer.
Set-WSManQuickConfig : <f:WSManFault
xmlns:f="http://schemas.microsoft.com/wbem/wsman/1/wsmanfault" Code="2"
Machine="localhost"><f:Message><f:ProviderFault provider="Config provider"
path="%systemroot%\system32\WsmSvc.dll"><f:WSManFault
xmlns:f="http://schemas.microsoft.com/wbem/wsman/1/wsmanfault" Code="2"
Machine="sv7731"><f:Message>Unable to check the status of the firewall.
</f:Message></f:WSManFault></f:ProviderFault></f:Message></f:WSManFault>
At line:65 char:17
+                 Set-WSManQuickConfig -force -SkipNetworkProfileCheck
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   + CategoryInfo          : InvalidOperation: (:) [Set-WSManQuickConfig], InvalidOperationException
   + FullyQualifiedErrorId : WsManError,Microsoft.WSMan.Management.SetWSManQuickConfigCommand

PS C:\Windows\system32> Enter-PSSession -ComputerName localhost

Enter-PSSession : Connecting to remote server localhost failed with the
following error message : Access is denied. For more information, see the
about_Remote_Troubleshooting Help topic.
At line:1 char:1
+ Enter-PSSession -ComputerName localhost
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   + CategoryInfo          : InvalidArgument: (localhost:String) [Enter-PSSes
  sion], PSRemotingTransportException
   + FullyQualifiedErrorId : CreateRemoteRunspaceFailed

----------------------------------------------------------------

Note "Unable to check the status of the firewall." part of the message.
Why would Enable-PSRemoting command try to check the status of the firewall?

Another Windows WTF reported by yatur


Update
Finally we were able to fix this remote powershell issue.
The problem was in Group Policy for IpV4Filter on our production machine.
IpV4Filter was limited to a single IP address (main address of that production machine).
I have no idea why it was setup that way.

This is how I fixed WinRM localhost access problem:

Run gpedit.msc
Local Computer Policy
Computer Configuration
Administrative Templates
Windows Components
Windows Remote Management (WinRM)
WinRM Service
Allow remote server management through WinRM

In "IPv4 filter:" change "208.43.198.72" to "*":
IPv4 filter: *

In the end, PowerShell and Microsoft server tools leave a negative impression due to bugs and pathetic diagnostic.

Consider another PowerShell surprise:
"ls" and "dir" commands produce empty output in case when folder is empty. No headers, no message that says there are no files. Just nothing. WTF?