Process for updating the mapping of PPMI SomaScan sequence IDs to human genes

Hi everyone! I wanted to share my process for remapping PPMI’s proteomic data from the original gene table to the most recent information from SomaLogic. This is based on guidance provided by @tiago.azevedo after his discussion with a SomaLogic representative, as described here: https://rcop.michaeljfox.org/t/best-way-to-match-somascan-sequence-ids-to-gene-ids/1663.

Motivation

The core issue is that PPMI originally used a custom panel based on v3 of the SomaScan assay, with a corresponding table mapping aptamer sequence IDs to genes. Since then, SomaLogic have refined their assay and understanding, and they provide a website for looking up the mapping between a sequence ID and gene. When you look up one of the original PPMI SomaScan sequence IDs in the new website, there are many discrepancies with the original mapping table. The guidance provided by SomaLogic is that the sequence IDs immutably refer to the same aptamer across all assay versions, and we should be using the most recent gene mapping, as it represents the state-of-the-art understanding of which proteins these aptamers truly bind to.

However, it was not immediately obvious to me how to remap the sequence IDs for highest accuracy! This post documents my process, in the hope that it can be useful to others, and/or I can get some feedback to improve it further!

Datasets required for mapping

First up, I needed a bulk download of the new SomaLogic sequence ID to gene mapping. I found this from the SomaLogic GitHub page, where there is a tab-separated file of sequence ID, Entrez ID, and target description.

Next, I needed a mapping from Entrez ID to Gene Name, which I got from BioMart by selecting “Ensembl Genes 116” then “Human genes”, and selecting the “Attributes” of:

  • “Gene stable ID”
  • “Gene name”
  • “NCBI gene (formerly Entrezgene) ID”
  • “NCBI gene (formerly Entrezgene) accession”

(Note: this takes care of the problem where some IDs map to mouse genes. I’m only including human genes in the set of possible genes to map to!)

Finally I needed the list of PPMI sequence IDs. I specifically work with Project 151 data, which is CSF proteomics for all participants at the baseline visit. The PPMI Project 151 file from LONI has a second sheet called “Lookup”, with columns for LABEL, GENE_SYMBOL, and TESTNAME, among others.

Creating a mapping table

The first step was to join the first two datasets using Entrez ID. Using Pandas in Python, this looks like:

df1 = pd.read_csv("SomaScan_11k_seqIDs_EntrezIDs_targFullNames.tsv",
                   sep="\t", header=None,
                   names=["Soma ID", "Entrez", "description"])
df2 = pd.read_csv("mart_export.csv")

df2 = df2[df2["NCBI gene (formerly Entrezgene) ID"].notnull()]
df2["NCBI gene (formerly Entrezgene) ID"] = df2["NCBI gene (formerly Entrezgene) ID"].astype(int)

df1 = df1.merge(df2, how="left", left_on="Entrez",
                right_on="NCBI gene (formerly Entrezgene) ID").drop(
    columns="NCBI gene (formerly Entrezgene) ID")

Now we have SomaLogic sequence ID mapped to human gene name, where such a mapping exists through Entrez ID. Most of these genes have a “Gene name”, but there are some with a null value for “Gene name” which have a name in the “NCBI gene (formerly Entrezgene) accession” column. Next I create a “New gene name” column which takes the value from “Gene name” if it exists, but falls back to the Entrezgene accession if “Gene name” is null.

df1["New gene name"] = df1.apply(
    lambda row: row["Gene name"] if not pd.isnull(row["Gene name"]) \
             else row["NCBI gene (formerly Entrezgene) accession"], axis=1)
print(df1["New gene name"].nunique(), df1["New gene name"].isnull().sum())

This shows that we now have 9550 unique gene names, and 59 that are still null. Unfortunately there is not much I can do with those nulls, so they end up removed from the final mapping.

Finally, one sequence ID can legitimately map to multiple genes. One example is a protein complex comprised of components assembled from multiple genes. The original PPMI table handles this by listing multiple genes separated by semi-colon, so I did the same here. The logic is that we group by Soma ID, take as many "New gene name"s as there are, remove nulls, sort by alphabetical order, and join with a “;” character:

new_map = df1.groupby("Soma ID")["New gene name"].unique().apply(
    lambda arr: ";".join(sorted([a for a in arr if not pd.isnull(a)]))).to_frame().reset_index()

Now we have a table mapping SomaLogic sequence ID to a string of new gene name values. The head of the table looks like this:

And some of those with multiple mappings look like this:

Performing the mapping

Next I want to join these new gene name mappings to the original PPMI sequence IDs.

The PPMI TESTNAME is SomaScan sequence ID with an underscore and an extra number on the end. For example, the entry 12690-33_3 is sequence ID 12690-33, and 3708-62_1 is sequence ID 3708-62. We can create a new column for sequence ID as follows:

df3 = pd.read_excel("ppmi_project_151.xlsx", sheet_name="Lookup")
df3["Soma ID"] = df3["TESTNAME"].apply(lambda s: "_".join(s.split("_")[:-1]))

For ease of understanding, I added a column which indicates whether the Soma ID was in the new 11k mapping file or not (some older sequence IDs have been retired, so they no longer appear in the 11k map):

df3["In 11k?"] = df3["Soma ID"].isin(df1["Soma ID"].unique())

And lastly, perform the join to the new map, and write it out to a new file:

df3 = df3.merge(new_map, how="left", on="Soma ID")
df3.to_csv("ppmi_project_151_lookup_remap_oct2026.csv", index=False)

The file I generated by following this process is attached here. NOTE: there is no PPMI participant data here; this is purely the “Lookup” sheet with extra columns for “Soma ID”, “In 11k?”, and “New gene name”.

ppmi_project_151_lookup_remap_oct2026.csv (571.5 KB)

Analytics

The original PPMI lookup table mapped all 4785 sequence IDs to gene names, but this new mapping includes 377 null gene names. These seem to be due to retired SomaScan IDs, as “In 11k?” is False, which suggests SomaLogic are no longer confident in the precision or accuracy of those aptamers. As a result, I have left them blank for my own analysis (ie removed them), instead of using the old gene names.

When comparing the old mapping to the new mapping, 4221 Soma IDs map to the same gene name(s) as before (out of the 4785 total, or 4408 with a non-null new name). That means only 187 are still in use, but map to different genes than before. And the majority of these appear to be refinements to an existing name, instead of a wholly new gene, such as these:

However, removing those new null mappings seems to make quite a difference to my own downstream analysis. I end up with a much cleaner, smaller, more focused gene set than I was getting before, so updating the mapping was well worth it for me. It confirmed for me that I was getting the same signal as before, but it was a more refined signal with clearer focus.

Next steps

I’d love to hear any feedback on this process, or what results other have seen from updating the PPMI SomaScan mapping!

And huge thanks again to Tiago for kicking off this whole direction of enquiry!

Hi Vic, this is very cool, thank you so much for sharing your steps and the final CSV!

There are only two things I’ve done differently from you, I think worth sharing/discussing :slight_smile:

  1. Instead of using the txt file from the SomaScan.db repository, I used the annotation file that we can download from the SomaScan Menu after you register an account there. I did that because the Menu annotation file is dated from March 2026 whereas the repository txt file was last updated 2 years ago. The only problem is that I need to manually crop the Menu file such that I can load it in Pandas. I’m not sure whether it’s that much of a difference though because I also get 377 unmatched sequence IDs.

  2. For those 377 unmatched sequence IDs, the people I talked with at SomaLogic recommended to use the original annotation while they continue to look through internal data and documentation, and so that’s what I’ve done. However, having seen how the annotations changed since v3, I can see why one would just prefer to not use those values :slight_smile:

I’m only using this mapping for enrichment analysis, and so I just saved the mapping as a (pickled) Python dictionary for fast mapping, which of course is not ideal to share. I’m a bit busy these days but as soon as I get some time I’ll try to see how much difference in the mappings I got compared to yours (if any). I’m also thinking that the advantage of using the annotation file from SomaLogic’s Menu is that then I’m able to find (and remove) the 235 sequences that are now identified as being as Mouse, as well as include the 4 sequence IDs that were marked as Mouse in the original v3 annotations but have since been marked as Human. I don’t know how many of the 377 are included in these 235 sequences.

Apart from that, thank you so much for this! I’m also using the BioMart to get a mapping from gene entrez IDs to names (although I barely use that because I get almost all my enrichment databases as entrez gene IDs).

I’ll follow-up on my points with you as soon as I can :smiley:

Hi @tiago.azevedo, thanks for sharing the comparison to your approach!

Ah, I didn’t realize you could access a bulk download of the SomaScan Menu data by creating an account! If I get the time earlier than you, I will make the comparison with my first-pass mapping from the GitHub source of data, and report back whether there are any differences between the two versions. In particular, I’ll explore those 235+4 Mouse/Human sequences, to see if I can see any pattern or possible explanation.

It’s interesting that SomaLogic’s recommendation is to use the original annotations for those missing annotations in the new set, because they really do seem questionable! I can see the logic either way, but I think I will keep them out for my downstream analysis.

Once again, it’s super helpful to get your feedback on this approach, so thank you :smiling_face: :folded_hands:! I think by cross-checking the March 2026 SomaScan.db with the GitHub 2024 version, we’ll have fully explored the options for updating Project 151 with the most recent information available!

Hi again @vcatterson ! I had some time to look into this :slight_smile:

I mapped your Soma ID columns into Entrez gene IDs that I took from the Menu download page, and then I mapped into gene symbols using a file I had downloaded from BioMart (the BioMart file might be slightly different from yours but probably not by a lot). I did that in the following way:

def split_names(val):
    """ For example: 'CGA;CGB3;CGB7' -> {'CGA','CGB3','CGB7'}"""
    if pd.isna(val) or str(val).strip() == '':
        return set()
    return {x.strip() for x in str(val).split(';') if x.strip()}

def my_symbols(seqid):
    symbols = set()
    for e in seqid_to_entrez.get[seqid]:
        symbols |= entrez_to_symbols.get(int(e), set())
    return symbols  

vic_df = pd.read_csv('ppmi_project_151_lookup_remap_oct2026.csv')
vic_df['vic_names'] = vic_df['New gene name'].apply(split_names)
vic_df['tiago_names'] = vic_df['Soma ID'].apply(my_symbols)
vic_df['match'] = vic_df['tiago_names'] == vic_df['vic_names']
vic_df = vic_df.drop(columns=['TESTNAME', 'TYPE', 'UNITS', 'PROJECTID', 'PROJECT', 'ORGANISM', 'PI_NAME', 'PI_INSTITUTION']) # For easier visualisation
vic_df

The seqid_to_entrez dictionary is the one I created using the file from Menu - in the CSV you download they also have a “Entrez Gene Name” but because I prefer to use Entrez gene IDs every time possible, that’s the mapping I had; this dictionary also has entrez gene IDs from PPMI annotations based on the 377 sequence IDs not in SomaScan annotations. The entrez_to_symbols is the one I got from BioMart (I saved the previous mapping as strings and that’s why I need to have the int() cast first.

Here’s the resulting CSV in case you want to have a look: matches_vic.csv (351.0 KB)

There are 3 mismatches where sequence IDs were mapped to different proteins:

The remaining 373 mismatches we have are sequence IDs which you couldn’t find any gene symbol but my mapping mapped to gene symbols using the process above. Upon further analysis, this can be explained with:

  1. 230 sequences are coming from what is now identified as Mouse organism, which seems to indicate that SomaLogic removed mouse sequences before uploading to GitHub (technically the “tiago_names” column has values for these cases but just because of the BioMart automatic matching, not because I’m actually using these sequences)
  2. 143 sequences are not present in the Menu downloaded file, and the mismatch only comes because of, once again, the flow above using the “sequence > gene ID > gene symbol” mapping automatically.

That’s it, at the end not that big of a difference :slight_smile: Let me know your thoughts!

EDIT:
After writing all this I decided to also check what happens if I don’t include any mappings for the 377 sequence IDs that are not in the 11k Menu annotation (ie, removing them from my seqid_to_entrez). In this case, the only mismatches are the 3 proteins that I have in the image above. So, it seems that except for those 3 proteins, our mismatches come from proteins that are already in this grey area that it’s not clear what to do with them :slight_smile:
However, if we are to provide an “updated” CSV like what you were doing, we could use the Menu downloaded file to also provide an updated information about the organism of the sequence IDs.

Hi @tiago.azevedo, and many thanks for continuing with this work! I really like your approach, and it seems like there are some minor differences between the versions of SomaLogic’s mappings on GitHub and on the SomaScan Menu download.

Does this summary sound accurate to you?

  • There are 3 Sequence IDs which map to different genes in the GitHub version versus the Menu download (IL21R/IL18, PAPPA/PRG2, and PRTN3/“PRTN3;SERPINA”)
  • There are 230 Sequence IDs which now map to Mouse
  • There are 143 Sequence IDs which are not included in the GitHub or Menu download.

As you say in the “Edit” box, it seems like those first 3 are the most important, as there is reason to exclude the second two categories from downstream analysis. (What would it mean to include counts of Mouse proteins in human CSF? And we can assume the third category have been retired by SomaLogic due to inconsistencies or difficulties with the data from those reagents.)

But since there are still some updates in the Menu download that are not reflected on GitHub, this shows that we should use the Menu file to reflect the current state of understanding. This is a bit annoying, as I have been thinking about @hirotaka 's suggestion of making an automated notebook to add to the Useful PPMI Codes repo, but automating this mapping to run against a GitHub file is a lot easier than automating the download of a file behind an account login! :sweat_smile: But for now, the static mapping file is a good start. I will prepare a new version with the Menu download, and upload here shortly.

Thanks again for your help with this, and for instigating this whole chain of inquiry!

I agree with your 3 bullet points to summarise our differences :slight_smile: I’m not sure whether we can assume SomaLogic decided to remove some sequence IDs due to inconsistencies or difficulties with the data from those reagents - my understanding is that that assay was developed specifically for Novartis so I can imagine maybe there were some specific sequences that SomaLogic might have decided to remove for business reasons? :thinking:

And following-up on your “what would it mean to include counts of Mouse proteins in human CSF?”, apologies for repeating myself but another advantage of the Menu downloaded file is that you can see those other mismatches in Organism information from v3 annotations (both ways!)

Finally, I agree it’s annoying we cannot fully automate this process because not only the file is behind an account login, and we actually have to manually crop the table inside the Excel to convert to a proper table :sweat_smile: But if you provide the instructions and then the final mapping result like here I think that’s already very useful! :smiley: