Monthly Archives: January 2017

The Drought Is Over

The drought is over. No, not the water drought, although the recent rainstorms means that only 50% of the State is still suffering.  Not since the Schoenberg Prism during the 2001-2002 season has the Los Angeles Philharmonic performed a work by Arnold Schoenberg on a regular subscription concert.  I blogged about this five years ago, and again in 2014, when a performance of the Schoenberg Violin Concerto was cancelled due to illness of the violinist.

But now it is really over.  Last night, Gustavo Dudamel conducted the LA Philharmonic in the Begleitungsmusik zu einer Lichtspielszene, Op. 34 and Piano Concerto, Op. 42.  Emanuel Ax was the soloist.  Ax loves the concerto and has performed it many times over the years, including back in 2001 during the Schoenberg Prism.  Ax recorded the concerto with former LA Phil conductor Esa-Pekka Salonen, but not with LA Phil, but the Philharmonia Orchestra.  This week’s performance may even available on livestream.  Mark Swed gave it a nice review in the LA Times.

My parents said that the pre-concert talk by Russell Steinberg was fabulous, devoted entirely to the Schoenberg.  Apparently he had the entire audience humming the opening lines of the concerto.

One thing no one has mentioned, I think, is the private “program” of the concerto, which was composed in Los Angeles in 1942.

Life was so easy
Suddenly hatred broke out
A grave situation was created
But life goes on

My grandfather was 68 years old when he wrote the piano concerto.  The entire world was in flames.  He had a young wife and three very young children (ages 10, 5 and 2).  The contrast between the serene, sunny lifestyle in his new home, and the conflagration raging in his old one, must have been stark.  You can hear that contrast in the concerto.  It’s a timely reminder that we gotten through tough times before.

Letter to DOJ Inspector General Michael E. Horowitz

Dear Inspector General Horowitz,

As the individual who filed suit to unseal the FBI search warrant against Hillary Clinton, I was very pleased to hear the news on January 12 that you would be leading an investigation into the FBI’s handling of its investigation of Secretary Clinton.  I had called for such an investigation on November 14, before we had seen the search warrant.  I am sure your investigation will be far-reaching and thorough, but as someone who has given a lot of thought to this, I would like to let you know some questions that I hope you will be able to answer.

It has been very difficult for those of us in the public to determine the timeline of events.  For the most part, we have to rely on unsourced statements in various contemporaneous press reports.  For example, on October 30, the press reported that the FBI had obtained a search warrant against Hillary Clinton to search Huma Abedin’s e-mails on Anthony Wiener’s laptop.  As far as I can tell, there was no official, public statement with this information, but the news was leaked out in some fashion.  The story turned out to be correct, but no one has yet disclosed the source of this leak, which was the first time in the history of the investigation that there was any suggestion that the FBI felt that there was probable cause to believe a crime had been committed by Secretary Clinton.  At the time, the final week of the election, the public had no way of knowing if the report was true, nor was there any official explanation of the basis for the probable cause allegations. In my view, and in the view of many others who have analyzed the data, this fed speculation against Secretary Clinton and resulted in a drop in her support that likely changed the outcome of the election, an entirely predictable result.  So, the questions I have are what exactly did the FBI do, why, and what were they thinking at the time?

A good starting point for an attempted timeline of events are the articles by Seth Abramson on Huffington Post.  See his reports from December 15, December 22, and January 17.  Seth sometimes speculates as to motives, but his timeline of events is based on many unsourced press reports, including stories from the Wall Street Journal and Washington Post.  We all are hoping that you can confirm or correct the basic facts of what happened, as well as who may have leaked the information to the press in each instance.

As to the main issue, we need to understand how it happened that the FBI publicly obtained a search warrant against a candidate for President within two weeks of the election, a warrant which changed the outcome of the election, but found nothing. Let’s go backwards from what we now know and identify key mistakes that need to be explained.

(1) Huma Abedin must have used her husband Anthony Weiner’s laptop for her webmail, gmail or yahoo mail account and it left some sort of cache on the laptop. As Orin Kerr pointed out, the FBI never should have looked at these e-mails at all, because they were not the subject of the initial search warrant that was looking for texts between Weiner and some specific teenage girl. So that was the first mistake.

(2) The FBI looked at the e-mail headers and saw there were e-mails between Hillary Clinton and her assistant Huma Abedin. It would be “probable” at that stage to infer that these were copies of the work-related e-mails that had already been reviewed, including ones Abedin had already voluntarily produced from her own laptop and blackberry. That would have been very easy to check, using the headers of a few e-mails. There is no evidence that the FBI tried to do that. That was their second mistake.

(3) The FBI could easily have asked Weiner, Abedin or Clinton for permission to review these new e-mails. They apparently did not. That was the third mistake.

(4) When FBI Director Comey then sent his October 28 letter, he said “the FBI cannot yet assess whether or not this material may be significant.” That was false, and it created the absolutely false impression that there was a possibility that the e-mails were not simply duplicates of ones already reviewed. All the FBI had to do was check the headers they had already reviewed against the e-mails they already had, which is something they clearly could have done. It ended up taking at most a week to do this.  So Comey’s letter was false, and misleading.  That was a fourth mistake.

(5) Director Comey sent the October 28 letter even though he was advised by DOJ not to do so. A fifth mistake.

(6) Then the FBI obtained a search warrant, suggesting there was “probable cause” to believe a crime had been committed, and omitting key facts from the affidavit. That was false. For example, you should ask the agent who prepared the warrant (whose name has not yet been disclosed), why he did not include any of the information already released to the public about the investigation of Abedin and her e-mails, which she had voluntarily disclosed. See https://vault.fbi.gov/hillary-r.-clinton,  Part 3 of 5, page 84 et seq, released September 2, 2016, discussing the interview of Huma Abedin and the production and review of her e-mails.  This was a sixth mistake.

(7) Although Director Comey was aware, and wrote to his agents, that “there is significant risk of being misunderstood,” he did not correct the misunderstandings that ensued for another nine days. That was a seventh mistake.

The honest thing for the FBI to have done would have been to first compare the e-mail headers. Then, if they really wanted to obtain or delete any duplicate classified e-mails from the laptop, they could easily have done so by asking permission from Weiner and Abedin. They could have also informed Congress that they had found duplicates of Abedin’s e-mails on Weiner’s laptop and were taking steps to secure them, reiterating that this would not likely change the conclusions made in July. This is in fact what was reported by Newsweek and WSJ a few days after Comey’s letters, but through confidential sources, and so it could not have dispelled the obvious misimpression caused by Comey’s October 28 letter.

If the Hatch Act means anything with regard to the FBI, it means that the agency should not take steps that are likely to influence an election. It certainly means that the Director should not publicly re-open an investigation and obtain a search warrant, alleging probable cause to believe a crime had been committed, within two weeks of an election, especially when no crime had actually been committed and the warrant results in no actionable evidence.

Serious mistakes were made, and I do not yet get the feeling that Director Comey understands this.  I trust and hope that your investigation will help the public, and the FBI, understand what happened. If there are not already laws to prevent this type of interference on the eve of an election, there should be.

Geni.com at 10 Years — The Challenges Ahead

On January 16, 2017, Geni.com (in my opinion the world’s leading family tree building program) will celebrate its tenth anniversary.  Previously, I have written extensively on the myriad advantages of Geni’s World Family Tree and answered all of the most common complaints of those who don’t yet understand the benefits of the program.  Today I want to address the areas where I believe Geni still needs improvement.  I do this in my individual capacity, obviously not on behalf of Geni, where I serve as a volunteer curator.

Just one year ago, Geni’s World Family Tree reached the 100 million profile milestone. Today, it is over 112 million.  That’s a tremendous amount of growth for such a large tree. By way of comparison, Geni’s annual growth is about the same as the total size of its non-profit competitor, WikiTree, which has just 13 million total profiles.  WikiTree is obviously too far behind Geni to ever catch up, and I’m not sure why anyone wastes his/her time on that platform.  But is Geni still the largest collaborative tree?

A few years ago, FamilySearch, a website operated by the Mormon Church, started its own collaborative tree, called Family Tree. Last year, FamilySearch claimed that Family Tree had over 300 million connected profiles, with 2.5 million being added each month.  If true, that would make the FamilySearch Family Tree by far the largest collaborative tree in existence, three times the size of Geni’s World Family Tree.  I am prepared to believe it, but I still have some doubts about the veracity of FamilySearch’s statistics.

One way to get a better feel for how the FamilySearch tree is progressing would be to see the statistics on mergers. Merging is one of the key features of any collaborative tree, because the idea is to merge all duplicates into one definitive profile. The tree is like a giant jigsaw puzzle, where everyone is working on the same puzzle. However, if you search in databases of non-collaborative trees, you’ll find lots and lots of separate trees that cover the same territory. For example, on Ancestry, searching exactly for Thomas Alva Edison born 1847, you find 613 public trees and 337 private trees. A similar search on MyHeritage finds 209 trees. FamilySearch Genealogies, a legacy collection of independent trees from before the move to a unified, collaborative tree, has 124 of them. But on Geni or Family Tree, you should find just one, definitive profile for the famous inventor (See Thomas Edison on Geni or FamilySearch.) The large collaborative trees were created by merging smaller trees together. On Geni, each person who adds a profile is retained as a “manager” after a merger. So when you see that over 1,000 people are managers of Charlemagne, that means that about 1,000 different trees got merged together at that point.

Both Geni and FamilySearch started out with a bunch of separate trees of varying sizes that were merged together and then improved with additions and corrections. Both sites allowed GEDCOM uploads for a while, and then cut them off and started to merge and clean up.  The result on Geni is that about 2/3 of all the profiles added to Geni are now part of the big World Family Tree.  The remaining 1/3 are on smaller trees that have not yet been connected or merged into the World Family Tree. On FamilySearch, out of 1.1 billion profiles, less than 1/3 are part of its collaborative Family Tree. Geni’s ratio of merges to additions is about 10%. There were 1,190,057 merges in the past year, compared to 12,031,845 added profiles. FamilySearch doesn’t publish statistics on its merges, but it claims that its users are adding about three times as many profiles as Geni, at a rate of about 30 million per year. Family Tree has 3.45 million contributors, out of 7.4 million total FamilySearch users.  On the other hand, Geni claims to have had over 11 million users since its inception, with about 4 million connected to the World Family Tree. To me, these numbers don’t seem to make sense — two collaborative trees, created in essentially the same fashion by about the same number of people, but with pretty wildly divergent results in terms of size. It could be that there are many more duplicates in Family Tree, or maybe FamilySearch includes some of its legacy collection Genealogies in the stats, which would be the equivalent of Geni adding in the 2.6 billion profiles and 80 million members of MyHeritage to its statistics. There also could be some flaws in the algorithm used to create the statistics. Geni benefits from feedback from its almost 200 volunteer curators, many of whom are very tech-savvy, and have helped identify bugs in the statistic features over the years. I am not sure that FamilySearch is getting the same type of help from its users, because its statistics may be less visible, or its active users may have less access to the staff.

So, although we are trying to compare apples to apples, we may be comparing apples to oranges. And this is true in one other important respect. FamilySearch offers much more than just a collaborative FamilyTree. It also offers free access to records and other data, much as Ancestry or MyHeritage do, although the latter charge a subscription fee for access.  Whether or not FamilySearch’s statistics for Family Tree are accurate, it has a great advantage over Geni in its ability to offer the ability to access and link to its massive records collection without charge. Geni users can get an equivalent feature only by paying for a MyHeritage subscription.

In my view, FamilySearch is Geni’s only serious competitor. The attraction of free records access, if coupled with a comparable tree-building program for a similar user base, could tip the scales in favor of FamilySearch, if it hasn’t already done so. No one should want to work on the second or third largest collaborative tree. Users will gravitate to the one that is the most comprehensive and accurate.

Accuracy is a key feature for collaborative trees. Ultimately, they become much more accurate and complete than individual trees. Never listen to people that don’t work on collaborative trees who tell you that the big trees are inaccurate. Trust me, their small trees are much, much worse, no matter how loudly they protest. It’s easy to imagine why that is. Large numbers of people working together can accomplish much more than someone working alone. That is why Wikipedia has put all the other encyclopedia publishers out of business. Crowdsourcing works, and it works especially well in fields like genealogy, where people compile data and sources for basic genealogical facts. If you want a complete biography, of course you go to a book painstakingly researched and written by an individual, but if you want to know who a person’s parents, siblings, spouses and children were, and the dates of their birth, marriage and death, you can go to a collaborative tree where all the busy bees work together to build the hive. It’s usually not rocket science, and if you have some expert curators to help with the tough spots, things work very well. As I have explained before, the fact that mistakes are easily found and corrected on collaborative trees is what makes them, over time, so much more accurate. You need those millions of eyes wandering through to find the mistakes.

Of course, it’s hard to measure accuracy in a family tree. But one way is to pay attention to what types of mistakes are being found and corrected. Those of us who are actively involved with Geni as curators get a pretty good picture, not only through our own work, but from all of the work we do to help other users resolve issues. When there are accuracy problems, we hear about them. The absence of complaints in a particular area is often a good indication that the product is becoming more accurate. This is also true because the Geni news feed alerts other users when work is being done in an area. That invites more eyes, and more mistakes are found, until everything is more or less correct, and people move on. Bees in a hive, ants in an anthill, that’s exactly how it works. You’d laugh at a bee working alone on a hive, or an ant trying to build his anthill by himself. It’s the same with genealogists. You have to just pity the poor folks who labor by themselves, even those who do so by looking at their neighbors and copying what they are doing (which is how most of Ancestry and MyHeritage users work). So, collaborative trees are the future, and the more users the better. But which one will be the winner? Where should we be putting our energies?

The competition between Geni and FamilySearch may come down to the quality of its curating team. Geni’s almost 200 volunteer curators are experienced genealogists from all over the world. They are usually people who have added 5,000 or more profiles to the World Family Tree, worked on projects, and been helpful to others in discussions. If Geni is to succeed, it needs to expand the number and range of its curators, to cover different parts of the world that are currently underserved. The recent surge by FamilySearch into the collaborative tree field should be a wake-up call for Geni to concentrate on maintaining its curating advantage and building on the strengths of its volunteer base.

Geni has many, many other features that currently give it an advantage over FamilySearch. DNA is a recent addition, and a significant advance that will only get better as more people upload their dna testing data.  Projects are also a huge resource, unique to Geni. Master Profiles are a great way of highlighting important people and well-sourced profiles. About 80 Languages are supported, and profiles can be multilingual.  Geni’s free Relationship Finder is phenomenal. When coupled with an additional paid MyHeritage subscription, Geni users can take advantage of matching to record collections such as newspapers and books that may not be available on FamilySearch. Geni users can also take advantage of matches to FamilySearch trees, thanks to a partnership between MyHeritage and FamilySearch.  In addition, FamilySearch may be more limited in its appeal because it is part of a religious organization. For example, gay marriage is not yet supported as a feature on FamilySearch, but is available on Geni.  And Jews and others have expressed discomfort over the Mormon practice of posthumous proxy baptism.

Nevertheless, there are a number of important areas where Geni needs desperately to improve if it is to maintain its position.  These are:

1. Geni uses Adobe Flash Player, which works fabulously and makes for a great user experience, except that Adobe has been in a battle-to-the-death with Apple since 2010, and may not come out of it alive. Presently, iPads and iPhones (43% of the U.S. smartphone market!) cannot use Flash and so Geni does not work well on those devices, relying on a rudimentary HTML5 version of Geni which pales in comparison to the Flash original. Geni needs to upgrade its HTML5 version ASAP.

2. Search on Geni is, in a word, atrocious. There is no wildcard or partial word (begins with, ends with, contains) search, no soundex or fuzzy search capability, no possibility of searching by town only. These are basic features for any genealogy program and it is an embarrassment that at age 10, Geni does not offer them. Most of us have to use Google to search public Geni profiles when we don’t know the precise spelling.

3. The matching algorithm needs to be upgraded. Geni has not been able to take advantage of MyHeritage’s Global Name Translation technology. As a result, matches are often missed by the algorithm and needless duplication occurs, as people blithely add new profiles, unaware that they are duplicating existing parts of the tree. Often, the algorithm misses exact matches on unique names, only because the surrounding profiles don’t match exactly.

4. When clicking on Research this Person, the MyHeritage results are often unusable. The search doesn’t take into account all the details on the profile (like dates), so the results are often not ranked properly. I often go over to Ancestry to search for records or other trees because their results seem more relevant, and it is much easier to adjust the search to make it tighter or looser.

5. Just as surname projects are created automatically, so should town projects. Every public profile with an event in a town should be automatically attached to the town project.

6. We should be able to sort and search project profiles.  Right now we have to page through them, or download an excel file.

7. Geni should integrate the SmartCopy program developed by curator Jeff Gentes as a Pro feature.

And now some more fantastic proposals:

8.  No one ever listens to me, but genealogy companies are the only companies in the world that hide from their customers. By making all living profiles private, you prevent living people from finding themselves on the tree when they google their own names. This is insane. At some point, some company will figure this out, ignore the paranoid hysteria of privacy nuts, and the rest will all go bye-bye. Until then, we should at least have the option of making unclaimed, living relatives public and searchable, so that we can increase the chances of having them find us. Some of us do genealogy because we actually want to find our family. Don’t we have a right to a program that serves our needs?

9. Geni should take more advantage of Facebook. It’s the biggest genealogical database in the world (1 billion names, many connected to family members) and yet no one has really harnessed it for genealogy. Wouldn’t it be nice if you could have Geni help find your relatives on Facebook?

10. Geni should add automated user stats like Wikitree does for Top Contributors. Let everyone know what the most active users are doing. A little competition isn’t a bad thing, and it also would allow curators to identify other active users. We have a manual version of Top 10 Lists in various categories, but an automated version would be much better.

11. Finally, my dream is that we could start getting the computer to do more of the work for us in finding matches. Right now, Geni just looks at its own data and the various data sets on MyHeritage. But there’s a whole world of big data out there that is not inside the proprietary walls. The next generation of genealogy company will take your tree and then scour the Internet looking for matches for you, wherever they maybe.

So, I am still bullish about Geni.  It is by far the best tree-building program available today. But things don’t stay the same. We’ll just have to wait and see what the future may bring.