dennisgorelik: (Default)
How-to create a VM on Hyper-V
Yesterday I learned how to setup a new virtual machine from scratch on Hyper-V.
Then I created an instruction for that:

-----
1) Install "Hyper-V" (if not installed yet)
https://superuser.com/questions/568425/hyper-v-virtualisation-disabled-in-firmware

2) Launch "Hyper-V Manager".

3) Create Virtual Network Switch (if not created yet)
- Right-click "RONAM" -> "Virtual Switch Manager".
- "New virtual network switch" -> "External" -> click [Create Virtual Switch].
- Make sure that "External network" is selected.
- Make sure that "Intel(R) Ethernet Connection (2) I219-LM" is selected.
- Click [Apply].
- "Pending changes may disrupt network connectivity" popup would show up. Click [Yes]. Wait for ~10 seconds.

4) Right-click "RONAM" and select "New" -> "Virtual Machine".

5) "Specify Name and Location"
Name: "BaseVM"
Location: "C:\VM"

6) "Specify Generation"
Select "Generation 2" radiobutton.

7) "Assign Memory"
Startup memory: 4096 MB 

8) "Configure Networking"
Connection: "VS2017Switch"

9) "Connect Virtual Hard Disk"
Select "Create a virtual hard disk" radiobutton.
Size: 80 GB.

10) "Installation Options" sub-tab
Select "Install An Operating System From A Boot CD/DVD-ROM".
Image file (.iso): "C:\install\en_windows_server_2016_x64_dvd_9718492.iso"

11) "Summary"
Click [Finish].

12) Right-click "BaseVM" -> "Start".
That would start Windows installation.
-----

Then I asked K. (a developer on my team) to test my instruction by creating a new VM from scratch and fix the instruction if needed.

The "fix"
K. successfully created new VM and "fixed" the instruction by adding small details to it. That effectively made that instruction about 25% longer:
Read more... )
Formally, that "fixed" instruction is correct -- clicking "Next" button is one of the likely steps that person would take in order to go through new VM settings.
However including "Click [Next]" steps into the instruction makes instruction worse and not better, and here is why:
1) The longer the instruction - the harder it is to read, understand, and follow that instruction.
It is also harder to review and modify longer instructions.
2) The obvious steps in an instruction - distract the reader from the non-obvious steps, such as "what 'Generation' option to choose" and "how much RAM to allocate".
3) Any user who is going to create a new VM from scratch does not really need to get instruction on how to operate the wizard:


Understand your readers
Here is a rule that K. violated by "fixing" the instruction:
Do NOT include into instruction steps that are obvious for all likely readers of that instruction: if the reader is able to reliably figure out an omitted step in a few seconds - then this step does not belong to the instruction.
However if the step is obvious only for some readers and is not obvious for other readers - then such step should be included into the instruction (because it is much more time consuming to figure out non-obvious step than to skip obvious instruction steps).

Understand your goals
Another important consideration when writing instructions - is to clearly understand the purpose of the instruction. I knew why I needed that instruction:
1) To help developers on my team to quickly familiarize themselves with setting up Hyper-V VMs.
2) In the future - to remind me and other developers key steps we used in creating our VMs.
3) To have a source file that we can edit to reflect key choices in configuration of our VM.

K. did not have these goals in mind and, actually, thought that such "VM setup" instruction is useless, but puffed up the instruction anyway. According to his belief - all instructions must be written in such a way that even a dummy would be able to follow it. The mistake here is to assume that the dummy (who does not know how to navigate a wizard) would actually read our instruction.
dennisgorelik: (Default)
Santhosh claims in his resume that he is a Web deveper.
Interview showed that he is probably a web or graphic designer. But not a developer.
He mentioned Javascript, but when I started to talk with him about specific task that could be implemented in Javascript - he quickly gave up.
All people on his team have "Senior System Analyst" title, but from my understanding, Santhosh is a junior at his role.
When describing the accomplishments, Santhosh used word "we". Sometimes "we" meant him, and sometimes "we" meant his team.
Skype audio connection was good (which is not typical for Skype calls to India). That is probably because Santhosh worked (on the bench) from his employer office.
2000 rupees per day ($30/day = ~$700/month).

Unfortunately nothing of what Santhosh can do a meaningful contribution to PostJobFree: we need mostly backend work (middle-tier, parser, SQL queries and database design) or solid UX. Santhosh did not show signs of either of that knowledge.
So I told Santhosh that his skills do not match what I am looking for and asked him if he has any questions for me.
He did not have any questions.
Few minutes later he messaged me:
Santhosh: Hi is any possibility to give one task related to Ui Ux Design and see if I didn't complete we can drop for further or else we will continue as well
Ui Ux/Front-end Development
Dennis: I do not have tasks suitable for your skills
This was a 27 minutes interview.
I should learn to recognize such mismatches much faster.
dennisgorelik: (Default)
Couple of days ago I interviewed Volodimir from Ukraine.
Volodymyr promised to work 6 days per week 14 hour per day for about $1500/mo.
His expertise is in writing "data processing" code.

I asked Volodymyr to give me examples of input and output of his processes.
Volodimir said that the input could be anything.
I asked him to be more specific, so I could understand.
Volodimir kept insisting that it could be any data.
I asked what kind of business problem does that process solve.
Volodimir kept insisting that it does not matter.
Eventually we both gave up in frustration.
I wrote to Volodymyr "your skills probably would not work out for working with me -- I simply would not be able to communicate with you clearly".
Volodymyr replied:
Dennis,
Look
This is a content of one column of one row of more than 1000000 rows which I use as input data : "2025050201401014016060 6090305025050201401014016060609030507014010901303016014".
If this is interested for You - try to understand what is this.
Your knowleges in programming is so low.
At first, You need to understand what is a main tasks of programming.
At second, You need to choose a tasks which You will solve and decide for why You need it.
You absolutely not understanding bases of programming.
When You will have enough skills in programming You will stop ask "an examples of data you are working".
I think - speaking skills of Russian, English or other languages for speaking about nothing - is just spent time. 
I`m usually very busy. 
And don`t want spent time.All Your conclusions is - big mistake.
I don`t want spent time for nothing.
Sigh.
dennisgorelik: (Default)
Early ElasticSearch History
Back in 2010 Shay Banon created first version of ElasticSearch.
Over the years the product matured.
In November 2012, ElasticSearch team received $10M in Series A funding.
Then in February 2013 they received $24M in Series B funding.
That helped them to produce very robust ElasticSearch 1.0 (2014-02-12) and then ElasticSearch 1.6 (2015-06-09) that we currently use.

$70M bloat
June 2014 - $70M Series C funding.
Shay Banon became a CEO and excused himself from active involvement in development and communicating with customers.
That is where the bloat began.
It looks like ElasticSearch team decided that since they have so much money - they can do pretty much whatever they want.
So they broke backward compatibility of their percolator by squeezing percolator into the standard format of ElasticSearch index.

What is percolator?
ElasticSearch percolator does reverse operation to a standard ElasticSearch query.
Standard ElasticSearch query allows our job seekers to find matching jobs.
Percolator allows job seekers to use their job search query in order to create a job alert.
Then when, in the future, new job is posted (by somebody else) -- the percolator is able to find all job alerts that job seekers created. That allows us to notify all owners of these matching alerts about new matching job (within a minute of receiving a job).

Differences between standard search query and percolator query
Because of the reverse nature of percolator, it functions very different from a standard search query:
1) Standard search query should normally produce only 10 results (users is unlikely to read more) and support paging.
Percolator always wants to get all matching alerts (also known as "percolator queries") - not just 10 of them, because every job seeker wants to get notified about new matching jobs to their favorite job alert.
2) Standard search - ranks search results based on the quality of the match (and then order results by descending rank). Such ranking does NOT make sense for percolator (because every job seeker wants to get notified anyway).

Why use standard search index format for percolator?
So why had ElasticSearch team decided to break backward compatibility and merge Percolator into a standard search index format?
This is their excuse:
---
https://www.elastic.co/blog/elasticsearch-percolator-continues-to-evolve
Prior to 5.0, all percolator queries need to be executed on this in-memory index in order to verify whether the query matches. So the idea is that the less queries that need to be verified by the in-memory index the faster the percolator executes.
---
In my first reading of that ambiguous claim I thought that ElasticSearch would be able to automatically detect what percolator queries is ok to skip, so it would, effectively, improve percolator performance.

What actually happened
We spend few days to setup proper experiment and found out that ElasticSearch 5.4 percolator is 3 times slower than ElasticSearch 1.6 percolator (or in other words, ElasticSearch percolator performance degrades proportionally to the version number).

The correct interpretation of that "less queries that need to be verified" claim actually meant that application developer in ElasticSearch 5.4 has an option to tag percolator queries (alerts), and then write code that would help percolator to skip alerts that have no chance to being triggered by a document we percolate.
But the problem is that it is very hard to come up with such "alerts skipping" algorithm. Percolator is so valuable in the first place exactly because of that ability to determine what alerts match and what alerts do not!

The summary
Series C $70M funding encouraged ElasticSearch team to break backward compatibility and produce useless features (such as paging and ranking in percolator) + degrade performance 3x.

Next: ElasticSearch Percolator Bloat - the Defense
dennisgorelik: (Default)
Couple of days ago a scam team, in order to sign up for premium membership on postjobfree.com -- hacked couple of real business emails: jdicken@c21patt.com and heidi@heidiburns.com
Both emails seems to belong to real estate agents.

I was able to find and reach actual owners of these email addresses on the phone.
They do not seem to be very concerned about their email addresses being compromised.
Both email owners confirmed that they did not make premium orders on our web site and then indicated that they prefer to continue with their other business instead of digging deeper into that email hacking ...

How much would you care if you learned that your email account was hacked?

dennisgorelik: (Default)
I got an email reply from a job seeker to our "Your Best Skills" email:
---
Hello Dennis,

As a black woman in computer science and engineering, I am confused as to why my top job skill in the email you sent me was listed as "black". Please shed some light on this as it is being preceived as quite insensitive. See the attached picture for reference.

Quick investigation showed that her resume explicitly lists:
National Society of Black Engineers
Black women’s Association scholarship
Graduate Advisor for the UCR National Society of Black Engineers (NSBE) Chapter

So I replied:
---
{NameRedacted},

You listed word "black" on your resume multiple times, so it was automatically added as your skill, since you put so much emphasis to it.

You are welcome to rearrange your skills here:
https://www.postjobfree.com/candidate-skills-profile

I already removed "black" skill from that list.

Does it help?
===

I considered removing word "black" from allowed skills list, but then remembered that there are such things as "black belt" (in management) and "black magic".

Besides, some job seekers may actually target jobs where they can reap benefits of affirmative action in big corporations.
dennisgorelik: (Default)
When developing a user-facing application, prioritization of security versus usability - requires delicate balancing.

Here are some examples:

1) When postjobfree emails to a user account recovery link, we want to make that the link is usable, and allow user to use that link for a day (24 hours).
In some scenarios such account recovery link could be usable for the user even after 24 hours. But such a long window for taking over an account based on a single link would be making postjobfree.com account less secure (what if an attacker get an access to an old "account recovery" link?).

2) When user opens that account recovery link, PostJobFree allows user to set a new password, and it also autologins user to that account.
But what if user opens the same link for the second time: should PostJobFree allow user to change account password and autologin again or not?
From security perspective it is safer to expire such a link immediately after user opened the link.
From usability perspective it is better to allow that link to work for the second time, because user may accidentally open that "account recovery" link twice. Or an antivirus program may pre-open email link before user opens it.
In order to balance these security and usability demands, we decided to allow account recovery link to work for 1 hour after it was already used (unused account recovery link can be used for up to 24 hours).

3) What if user changed password on his account: should we allow old account recovery links to work or not?



Here is a typical "security" scenario:
User account owner found that an attacker (or a former employee) has an access to the account. So the account owner changes the password and expects that the attacker would not have an access to the account anymore. But if the attacker still has an old account recovery link - he can still autologin.
So, from security perspective, we should immediately expire all "account recovery" links that were sent before password change.
However there is an important "usability" scenario too:
- User posts a job, which creates a new account for the user.
- PostJobFree emails "confirm email" link to the user:
===============
From: PostJobFree <noreply@postjobfree.com>
To: testuser@gmail.com
Subject: Confirm your PostJobFree registration

===============

("Confirm email" link functions similar to "account recovery" link).
- While waiting for that "confirm email" link to arrive in the email inbox, user sets up a password on that new account (as a part of a new account setup process).
- Then user opens "confirm email" link.
If, according to security demands, "password change" in the previous step expired such "confirm email" link, then an important piece of usability is lost: user cannot immediately confirm that email is functioning, and has to request another "confirm email" link.
So, how do we balance these contradictory demands between security and usability in this case?
The best approach seems to be to prioritize usability in cases when user sets up a new account, but prioritize security when user changes password on already established account.
So if user changed password while going through initial account setup wizard - then keep previously sent links functioning. But if user already had password set, and now decided to change the password again - then expire all past "account recovery", "confirm email" and "change email" links immediately.
Such granular balancing between security and usability allows to deliver good security to the users who care about security of their account (users who change their account passwords manually -- such users are a minority), but still deliver a good usability to the vast majority of users who setup their password only if they are nudged by the account setup wizard.
dennisgorelik: (Default)
Couple of weeks ago we noticed that the same C# code executes differently under MSTest and in Visual Studio 2017.
In particular, Uri constructor crashed on invalid input in Visual Studio, but did not crash in MSTest.

Then, several days later, we found that ASP.NET allows to modify collection that we iterate through, but the same code crashes in a unit test with "System.InvalidOperationException: Collection was modified; enumeration operation may not execute".

We decided to investigate and found that the culprit is in a different value of "httpRuntime targetFramework" attribute.

Bad naming and documentation
Microsoft .NET Framework team chose a bad name for that attribute and wrote a misleading documentation:
---
https://msdn.microsoft.com/en-us/library/system.web.httpruntime.targetframework(v=vs.110).aspx
The version of the .NET Framework that the current web application targets.
---

When most developers (including me) read that - they think that "targetFramework" attribute defines what version of .NET framework would execute.

But actually that attribute has a very different meaning and should have been named either compatibilityTargetFramework or quirksTargetFramework.

What httpruntime targetframework actually means
Fortunately, levibroderick wrote a clarifying blog post, that now is the first result for httpRuntime targetFramework search:
https://blogs.msdn.microsoft.com/webdev/2012/11/19/all-about-httpruntime-targetframework/

With new versions of .NET framework, Microsoft .NET team introduced some breaking changes (especially for .NET Framework 4.5).
So then they created "quirks" to fix these breaking changes.

So, "targetFramework" attribute pretty much defines what set of quirks to use (the older the targetFramework version is - the more quirks you would get).
The total number of quirks seems to be around 10 (could be a little bit more or less, but not by a lot).

Practical impact
In the past, our Web.Config did not contain any mentioning of targetFramework in <httpRuntime> element.
That meant that we got all the quirks, so postjobfree.com did not break.
Then yesterday we turned off "legacy compatibility mode" by setting
<httpRuntime targetFramework="4.6.2" />
We lost all the quirks that way and, as a result, got two bugs:
1) "WebForms UnobtrusiveValidationMode requires a ScriptResourceMapping for 'jquery'. Please add a ScriptResourceMapping named jquery(case-sensitive)." crash on every page that contains <form> element.
2) Encrypted validationKey in <machineKey> element changed its meaning, so all users authentication cookies expired.
Several hours of research and development later - we fixed these issues and now our web site runs in a quirks-free mode.

What was your experience in converting legacy .NET app to the new .NET Framework version?
dennisgorelik: (Default)
We host 2 PostJobFree servers on SoftLayer (in their Dallas datacenter).
In the last year I started to get more and more warning signs that SoftLayer is slowly decaying (after acquisition by IBM 3 years ago).
So, finally, I decided to check how good is uptime of https://www.softlayer.com/

So I created a new "Keyword" monitor on https://uptimerobot.com
The monitor checks if "Data Centers" wording was rendered into SoftLayer's home page HTML.
UptimeRobot runs that check every minute.

So, how much uptime does the legendary hosting is able to keep for their web site?
According to UptimeRobot, SoftLayer's home page uptime is a pathetic 99%.
That means that there is 1% change that Softlayer home page is down at any given moment.
Among hosting providers, uptime below 99.9% is considered poor, and uptime above 99.99% is considered good.

According to UptimeRobot, when SoftLayer's home page is up, it has average response time of 681.72ms (about 0.7 seconds, which is kind of OK).

To put things in perspective: PostJobFree home page (that is hosted on dedicated server in SoftLayer) has 100% uptime (99.99%+) and 139ms average response time.



So for now our dedicated servers on SoftLayer still work, but if SoftLayer tech team keep deteriorating, they would eventually mess up their core network too, and then it would bring downtime to our servers as well.

So I am looking for a new hosting provider now.
Would you recommend any?
dennisgorelik: (Default)
This fraudulent job was posted from Ukrainian IP address (195.211.155.21) to United Arab Emirates location:
------------
Receptionist,Security guards and drivers at Mount Sinai Hospit MOUNT SINAI HOSPITAl 6000 we are in need of the following workers
Doctors
Nurse
Dermatologist
General Practitioner
Translators
Accountants
lab technician
Hospital driver
Pharmacist
Receptionist
Dentist
Security man and many more
get back to me with your resume .it is interesting to know that there is an added allowance upon the salary
------------


After one of candidates applied to that job, the scammer emailed to the candidate:

March 27, 2017 3:00 PM, infomount sinaihospital <infomntsinaihospital1@gmail.com> wrote
=================
Dear Candidate,

The human resource department of Mount Sinai Hospital USA
welcomes you once more. We have received and read through CV and the responses presented with satisfaction. After reading through them .we have accepted to employ you in the hospital for the vacancy you applied for .We also made this decision based on your
resume and partly on the answers to the few questions. Also, we will
like to let you know that employees must arrive here in the United States one week before resumption date for proper orientation. We hereby present to you the general terms and conditions under which you have been employed

GENERAL TERMS AND CONDITIONS

1) SALARY: Salary allocated for this position is $5400 USD monthly.
i) ALLOWANCES: You are entitled to $100 USD weekly allowance
exclusive from salary.
ii) SALARY INCREMENT: You shall have a 5% salary increment at the end of the first six months of effective service.

2) ACCOMMODATION/ ENTERTAINMENT: The Company assists you by paying up to 50% of your consummation bills. The accommodation includes a two room apartment with internet facilities, toilet and kitchen fully equipped not too far from the hospital(just a walk)

3) WORKING HOURS, RESTING DAYS AND HOLIDAYS: You will work 6 days a week, (54 hours) . Work starts from 8:00 A.M. to 12:00 AM and 1:00 PM to 5:00 PM. You will be free all through Sundays as from 10:00am. You shall have a two week break after every 5 months

4) OVERTIME WORK AND COMPENSATION: For any activity carried out outside the work time schedule above, you shall be compensated in accordance with the labor code.

5) OBLIGATIONS: You are obliged to do your best at work especially during the hours of work.

6) DURATION OF CONTRACT: The contract will have a duration of two years and may be renewed or terminated upon the decision of one or both parties with justified reasons

7) BENEFITS: Employees shall be registered with the company's health
assurance policy, have a one week trial period upon your arrival at
the company during which you shall be compensated according to the
labor code

8) UNIFORM: Employees shall appear in the Hospital’s official uniform

9) CONTRACT SIGNING: A contract document shall be sent for you to sign if you agree to the terms and conditions of the job

10) EMPLOYMENT POLICY: Our employment policy states that all foreign employees shall travel with an H-1B work visa and shall pay for the cost of processing of their travel documents (work Permit, Visa etc.) for security purpose. Since all applications for H-1B work visa must pass through the United States Citizenship and immigration services (USCIS) and the American immigration lawyers association (AILA) for approval before a visa a endorsed, our Hospital‘s lawyer barrister Ayoub who is a member of this association will be responsible for the follow up of all the processes until a visa is due for you at the United States embassy in your country. You are required to pay $250 directly to the lawyer for all the paper work and other expenditures. The hospital shall be responsible for your flight ticket after your visa has been endorsed. It should be noted that before you can travel to the United States and start work, you will need some documents which are required for your entry and work in here. The documents are:

Social Security Number (SSN)
Migration clearance
Work Permit
Residential Permit
Contract form
Visa

11) REQUIREMENTS: The following documents and information shall be required from you in order to process the above required documents which will facilitate your relocation to the United States


- A scanned copy of your passport
- Birth certificate (optional)
- 2 color passport size photos
- Your current home address, city and country
- Your Contact Phone Number

The above information shall be used to process all documents
necessary for your travel and work. While hoping that the
above terms and conditions are satisfactory, we await your response
with the requested documents and information. Thank you

--
[]
Jane maksoud

Human Resource Department

Mount Sinai Hospital
One gustave l. Levy place
New york, ny 10029-6574
+1 315-512-2319
=================
dennisgorelik: (Default)


A recruiter told me that when he interviews candidates - he asks them their salary history.
Not just what candidate is making now, but what candidate has been making on his previous positions too.

There are two main reasons why he asks salary information:
1) Make sure that hiring company is able to pay what candidate was making.
2) Check consistency of candidate's story (does claimed salary look similar to the typical compensation for the position like that?).

I asked him: "Do you give that salary history information to the hiring manager?"
He said: "No. There is no need to pass that information to the hiring manager".

I asked: "If candidate had salary that is significantly lower than current position pays - do you reduce the salary that company offers?"
He said: "No. Company already has compensation defined for that position."

I asked: "How many candidates agree to share salary history information?"
He said: "Almost all candidates talk about their past salaries."

That openness is a little bit surprising.
I myself, in the past, did not share my salary information with recruiters (but I shared how much I want to make at the new position).

Do you share your salary history when you talk with recruiters?
dennisgorelik: (Default)
I asked a developer on my team to commit code frequently: at least once per day or more.
Even if feature is not ready - commit parts of this feature, prototypes, or even mixture of draft code and draft notes.

The developer objected that:
1) Not every feature fits into one day of development. (I agree with that).
2) Commiting raw code, and especially committing raw notes - effectively publishes them.
And publishing raw ideas - makes them to prematurely solidify, which, in turn, negatively affects future research, because there is a bias toward solid ideas, while alternative solutions are more likely to lose even if they are good.
(I kind of agree with that effect, but only partially. Publishing ideas solidify them a little, but that effect is more positive than negative).
3) Commiting/publishing raw notes violates developer's privacy, because raw notes represent unfiltered train of thoughts.
(My thinking is that since these notes were created as an attempt to find a solution to a work problem - there raw notes are very unlikely to contain any personal secrets. Raw notes can still contain weird thoughts or even silly mistakes, but that is OK - there is no need to be ashamed of mistakes in early research thoughts).

My main reasons to have code committed frequently is:
1) Check that work goes in the general direction that is likely to satisfy business goals.
2) Review new research and code in reasonably small chunks, so that code reviewer is able to understand them.
3) See not only final polished solution, but also mistakes and research dead ends along the way. That helps learning more.

What do you think: is trying to commit code at least once per day - too frequent or a good goal?
dennisgorelik: (Default)
For many years already I am using Skype Phone (1-904-425-9555).
I receive incoming calls from all over the world, call to the US and Canadian numbers every day and occasionally call to international phone numbers (India, UK, Russia, Australia, South Africa, Nigeria, etc.)

Skype charged me less than $90 per year.
Here are my transactions for the last year:
--------
Skype Number, 12 month subscription $40.12 (charged in February 2017)
Unlimited US & Canada 12 months $26.88 (charged in February 2017)
Skype Credit (for outgoing calls to other countries) $10.00 (charged in June 2016)
--------

Skype is a little bit messy with their billing (not immediately clear what "You sent a payment of $26.88 USD to Skype Communications Sarl
(paypalpayments@skype.net)" mean), but after finding out how much does Skype Phone actually cost to me - it seems like a good deal.

What do you use for your phone?
dennisgorelik: (Default)
In the last couple of days I learned quite a bit about multithreading:
1) How to create new thread in order to fix hangs in crawler.

2) That creating new threads has performance penalty (about 10 ms 0.15 ms per creation of a new thread).

3) That Task has good performance (almost no performance penalty) because it reuses thread pool.

4) That if you use Task (thread pool) you cannot really kill the hanging thread, so using Task Factory does not really help in solving a hanging thread issue.

5) How to create our own thread factory that is running single thread and how to kill that thread if it runs for too long.
In particular:
- How to use two EventWaitHandle objects in order to communicate appropriately between main thread an background/worker thread.

- When it is the right time to exit from the infinite loop in background thread in case if service is shutting down or pausing. (Exit background thread in case of pause/shutdown only if it is idle - to prevent confusion in the business logic of the client code).

using System;
using System.Runtime.ExceptionServices;
using System.Threading;

namespace PostJobFree.Utilities
{
	public static class ThreadHelper
	{
		private static EventWaitHandle CompletionWait;
		private static EventWaitHandle InputWait;
		private static Thread CurrentThread;
		private static Exception ThreadException;
		private static Action ActionToExecute;
		private static readonly object LockObject = new object();

		public static bool ExecuteOnSeparateThreadWithTimeout(Action action, TimeSpan timeout)
		{
			lock (LockObject)
			{
				ThreadException = null;
				ActionToExecute = action;
				if (CurrentThread == null // First-time execution or previous thread was aborted due to timeout
					|| !CurrentThread.IsAlive) // Service was paused
				{
					InitializeCurrentThread();
				}
				InputWait.Set();
				if (!CompletionWait.WaitOne(timeout))
				{
					CurrentThread.Abort();
					CurrentThread = null;
					return false;
				}
				if (ThreadException != null)
					ExceptionDispatchInfo.Capture(ThreadException).Throw(); // To preserve stack trace
				return true;
			}
		}

		private static void InitializeCurrentThread()
		{
			CompletionWait = new EventWaitHandle(false, EventResetMode.AutoReset);
			InputWait = new EventWaitHandle(false, EventResetMode.AutoReset);
			CurrentThread = new Thread(WorkerThread);
			CurrentThread.Start();

		}
		private static void WorkerThread()
		{
			while (true)
			{
				if (InputWait.WaitOne(TimeSpan.FromSeconds(1)))
				{// Action is requested
					try
					{
						ActionToExecute.Invoke();
					}
					catch (ThreadAbortException)
					{
						return;
					}
					catch (Exception ex)
					{
						ThreadException = ex;
					}
					finally
					{
						CompletionWait.Set();
					}
				}
				else
				{// 1 second passed with no Action request
					if (ExecutionCore.NewExecutionAllowed) continue; // To allow exiting on Pause
					return; // Exit on (NewExecutionAllowed = false) only if background thread has nothing to do
				}
			}
		}
	}
}

using System;
using System.Threading;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using PostJobFree;
using PostJobFree.Utilities;

namespace TestIjSearch.Utilities
{
	[TestClass]
	public class ThreadHelperTests
	{
		[TestMethod]
		public void ThreadHelperExecuteOnSeparateThreadWithTimeoutTest()
		{
			Assert.IsFalse(ThreadHelper.ExecuteOnSeparateThreadWithTimeout(() => Thread.Sleep(10000), TimeSpan.FromTicks(1)));

			Assert.IsTrue(ThreadHelper.ExecuteOnSeparateThreadWithTimeout(() => { }, TimeSpan.FromSeconds(1)));

			bool exceptionHappened = false;
			try
			{
				ThreadHelper.ExecuteOnSeparateThreadWithTimeout(() => {throw new PostJobFreeException();}, TimeSpan.FromSeconds(1));
			}
			catch (PostJobFreeException)
			{
				exceptionHappened = true;
			}
			Assert.IsTrue(exceptionHappened);

			try
			{
				ExecutionCore.NewExecutionAllowed = false;
				int result = 0;
				Assert.IsTrue(ThreadHelper.ExecuteOnSeparateThreadWithTimeout(() => { result = 111; }, TimeSpan.FromSeconds(1)));
				Assert.AreEqual(111, result);
				//Thread.Sleep(TimeSpan.FromSeconds(2)); // It allows asynchronous thread to die/exit if (NewExecutionAllowed = false)
				Assert.IsTrue(ThreadHelper.ExecuteOnSeparateThreadWithTimeout(() => { result = 222; }, TimeSpan.FromSeconds(1)));
				Assert.AreEqual(222, result);
			}
			finally
			{
				ExecutionCore.NewExecutionAllowed = true;
			}
		}
	}
}

Update: LJ discussion.

Unit tests

Feb. 2nd, 2017 10:27 pm
dennisgorelik: (Default)
How do you convince a senior developer to write unit tests even when he thinks that his code is easy to understand?

I already tried:
1) Unit tests have test cases that help to understand the underlying business logic during code review.
2) Having automated test coverage makes code more maintainable, because tests allow quick experiments with changing underlying implementation.
3) Tests are needed anyway (manual or automatic), so why not immediately write auto-tests during development?

Update: LJ discussion.
dennisgorelik: (Default)
A job seeker asked me if "Reliance Capital Limited" company that contacted her is a legitimate employer.
Judging by the company name and the way they communicated with her (text-only) - it probably is a scam.

But in order to find out for sure if recruiter is real deal:
1) Call them (scammers frequently try to avoid talking on the phone and hide behind text messages and emails).
2) Assume that there could be a scammer on another end and do not reveal your sensitive personal details.
3) Typical signs of scammers:
- Bad phone connection quality (because scammers frequently use internet proxy).
- _Heavy_ foreign accent from a poor country (typically Nigerian accent, but occasionally could be Russian or some other accent). The scammer would may insist that they are in the US or in London.
- Incoherent business story (ask them what they sell to their customers).
dennisgorelik: (Default)
PostJobFree crawler found web page that causes fatal crash in AngleSharp parser:
using AngleSharp.Parser.Html;
.....
string pageHtml = LoadUrlContent("http://onestop.fiu.edu/financial-aid/loans/")
var parser = new HtmlParser();
var document = parser.Parse(pageHtml);
document.QuerySelectorAll("a"); // Fatal crash: "An unhandled exception of type 'System.StackOverflowException' occurred in AngleSharp.dll".

We cannot catch that exception and it simply restarts the whole process (PostJobFreeService Windows service).
That is very frustrating.

In development environment that crash is not always reproducible.
When we run code above in test - it just works.
But if we run the same code under Visual Studio debugger - it crashes with 'System.StackOverflowException'.

Update:
https://github.com/AngleSharp/AngleSharp/issues/523
AngleSharp library maintainers noticed that problematic page contains a lot of "<content /><content /><content /><content />" attributes.
view-source:http://onestop.fiu.edu/financial-aid/loans/

Obviously it is not an excuse to fail. Hopefully their latest build would fix the problem.
dennisgorelik: (Default)
Today Andrey and I discovered that NeuralCrawler we created - got brain cancer: Out of 844,467 pages - 99.5% is useless junk from 2 sub-domains: "boystown.giftlegacy.com" and "boystowngift.org"

So far we attribute the cause of that cancer spread to a couple of bugs:
1) Creating extra links with every redirect (unfortunately problematic domains generate links with random sessionId and then redirect from one to another).
2) Not deleting old page links after reparsing page content.
dennisgorelik: (2009)
Today we switched postjobfree.com to https.
Now all pages are served over HTTPS, and if user requests http page - we redirect user to https://www.postjobfree.com/

Because all PostJobFree public pages are now secure, Google Chrome gives us "green lock" icon:
'Green lock' icon in Google Chrome

The reason to stop supporting legacy HTTP why we finally decided to make

This Google Blog article was the final nudge to switch from supporting both HTTP and HTTPS to "HTTPS-only" mode:
---
https://security.googleblog.com/2016/09/moving-towards-more-secure-web.html
Beginning in January 2017 (Chrome 56), we’ll mark HTTP sites that transmit passwords or credit cards as non-secure, as part of a long-term plan to mark all HTTP sites as non-secure.
---

We still support HTTP requests for some legacy feeds and APIs, but even that may stop in a couple of years.
dennisgorelik: (2009)
Our customer service send handpicked resumes to help our premium user with hiring for Family Service Counselors job.
Their reply:
---
The resume from {NameRedacted} that you submitted as a complimentary resume worked at our cemetery -
---

Profile

dennisgorelik: (Default)
Dennis Gorelik

July 2017

S M T W T F S
      1
2345678
9 101112 131415
16171819202122
23242526272829
3031     

Syndicate

RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Jul. 20th, 2017 12:37 pm
Powered by Dreamwidth Studios