[chimerax-users] How to get the average pTM scores from an Alphafold multimer prediction

Wed Jan 4 12:07:34 PST 2023

Hi Henry,

  The AlphaFold source code 

	https://github.com/deepmind/alphafold/blob/569eb4fea3733b979cb0442750b875759dd5ecc0/alphafold/model/model.py#L46 <https://github.com/deepmind/alphafold/blob/569eb4fea3733b979cb0442750b875759dd5ecc0/alphafold/model/model.py#L46>

shows that the ranking for multimer predictions uses 0.8 * ptm + 0.2 * iptm

	confidence_metrics['ranking_confidence'] = 0.8 * confidence_metrics['iptm'] + 0.2 * confidence_metrics['ptm']

while for monomer predictions it uses average plddt

	confidence_metrics['ranking_confidence'] = np.mean( confidence_metrics['plddt']) )

The ptm, iptm and plddt values are in the score json files, for instance, this score file for a multimer prediction with 191 amino acids

 	~/Downloads/ChimeraX/AlphaFold/prediction_62/af191_unrelaxed_rank_1_model_5_scores.json

contains

{
"max_pae": 31.75,
 "pae": [[...191*191 pae values]],
 "plddt": [87.08, 93.55, 96.38, 95.29, 97.09, 97.85, 97.47, 97.97, 98.56, 98.23, 97.81, 98.3, 98.41, 97.97, 97.66, 98.2, 97.63, 96.64, 96.8, 95.64, 94.65, 92.39, 95.56, 97.58, 98.3, 98.57, 98.53, 98.79, 98.72, 98.77, 98.72, 98.44, 98.03, 97.95, 98.07, 97.96, 98.57, 98.61, 98.73, 98.68, 98.72, 98.67, 97.64, 96.52, 97.08, 97.62, 98.44, 98.79, 98.72, 98.79, 98.56, 98.36, 98.1, 97.43, 97.71, 96.14, 97.7, 97.76, 97.2, 97.88, 97.88, 96.67, 97.31, 97.97, 97.85, 96.7, 96.35, 96.88, 97.01, 95.29, 95.22, 95.42, 94.62, 90.54, 78.83, 65.75, 53.81, 47.67, 44.63, 42.19, 43.68, 44.19, 42.65, 39.15, 35.82, 94.59, 96.97, 98.36, 97.91, 98.02, 98.02, 98.68, 98.8, 98.76, 98.7, 98.7, 98.75, 98.59, 98.33, 98.41, 98.56, 98.05, 97.69, 98.09, 97.51, 96.95, 97.35, 97.8, 98.5, 98.78, 98.82, 98.87, 98.65, 98.35, 97.35, 96.11, 92.05, 83.29, 79.73, 73.58, 71.57, 67.75, 65.78, 61.65, 60.54, 57.76, 57.54, 51.76, 48.72, 49.38, 47.89, 49.44, 50.32, 52.88, 55.32, 60.69, 77.64, 87.56, 94.38, 96.6, 97.72, 98.55, 98.74, 98.87, 98.74, 98.0, 96.52, 95.22, 96.96, 98.41, 98.71, 98.42, 98.34, 97.5, 96.99, 94.89, 93.43, 93.7, 95.99, 96.44, 96.43, 97.62, 98.36, 97.44, 97.34, 98.49, 98.53, 97.7, 97.71, 98.45, 98.32, 97.26, 97.75, 97.98, 98.11, 96.84, 97.43, 97.96, 97.35, 94.07, 85.21, 73.09, 59.76, 52.41, 50.31, 45.78, 47.12, 43.03, 43.64, 44.21, 35.19],
 "ptm": 0.83,
 "iptm": 0.8600000143051147
}

ChimeraX does not read this scores file but you could easily read it in Python, for example from the ChimeraX Python Shell (menu Tools / General / Shell)

import json
s = json.load(open('/Users/goddard/Downloads/ChimeraX/AlphaFold/prediction_62/af191_unrelaxed_rank_1_model_5_scores.json'))
s['ptm']
> 0.83

s['iptm']
> 0.8600000143051147

s['plddt']
> [87.08, 93.55, ...]

sum(s['plddt']) / len(s['plddt'])
> 88.15670157068061

The plddt values are also saved in the bfactor column of the AlphaFold PDB files.  So I could also get the average plddt by averaging the bfactor value of CA atoms using the Python shell

from chimerax.core.commands import run
run(session, 'select #1 at CA').atoms.bfactors.mean()
> 88.16

The PDB format only gives two digits past the decimal point in the bfactor column so they won't match exactly the values read from the json scores file.

   Tom

> On Jan 4, 2023, at 9:55 AM, Chen, Henry via ChimeraX-users <chimerax-users at cgl.ucsf.edu> wrote:
> 
> Is there a way to get the average all the pLDDT values for the entire complex? Isn't that how the program generates its ranking?
> 
> Thanks so much,
> Henry
> 
> -----Original Message-----
> From: Elaine Meng <meng at cgl.ucsf.edu> 
> Sent: Wednesday, January 4, 2023 11:40 AM
> To: Chen, Henry <hchen38 at illinois.edu>
> Cc: chimerax-users at cgl.ucsf.edu
> Subject: Re: [chimerax-users] How to get the average pTM scores from an Alphafold multimer prediction
> 
> Hi Henry,
> The per-residue scores in the PDB file are pLDDT values, not pTM.  I don't know if pTM is included in the information that ChimeraX gets. As described in the help links below, ChimeraX shows pLDDT and PAE (predicted aligned error):
> 
> <https://urldefense.com/v3/__https://rbvi.ucsf.edu/chimerax/docs/user/tools/alphafold.html*coloring__;Iw!!DZ3fjg!9Sn5HbsUfAGjOX6KCO4gTgqW8U-8KWx9hRu66Chltsb4p3nPBkXpVc9YKjWrZw7S6ZsdmYP3WbZvxLMCzXwS$ > <https://urldefense.com/v3/__https://rbvi.ucsf.edu/chimerax/docs/user/tools/alphafold.html*pae__;Iw!!DZ3fjg!9Sn5HbsUfAGjOX6KCO4gTgqW8U-8KWx9hRu66Chltsb4p3nPBkXpVc9YKjWrZw7S6ZsdmYP3WbZvxM3PnVJa$ >
> 
> I hope this helps,
> Elaine
> -----
> Elaine C. Meng, Ph.D.                       
> UCSF Chimera(X) team
> Department of Pharmaceutical Chemistry
> University of California, San Francisco
> 
>> On Jan 4, 2023, at 8:27 AM, Chen, Henry via ChimeraX-users <chimerax-users at cgl.ucsf.edu> wrote:
>> 
>> Dear Chimarax devs/users,
>> 
>> I’m using ChimeraX Alphafold tool to do a multimer structure prediction. I was wondering how can I determine the average pTM score for each structure prediction? I get PAE.png and plddt.png file, but not a text file or anything that tells me the average pTM. (I was hoping the ranking file would give the average score but it does not) The models come with the pdb file as well as the scores.json files, but again I don’t see anything that tells me the average pTM for the structure. Clearly the pTM per residue scores are saved in my pdb file, is there a command to average them all up?
>> 
>> Thanks!
>> Henry
> 
> 
> _______________________________________________
> ChimeraX-users mailing list
> ChimeraX-users at cgl.ucsf.edu
> Manage subscription:
> https://www.rbvi.ucsf.edu/mailman/listinfo/chimerax-users
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.rbvi.ucsf.edu/pipermail/chimerax-users/attachments/20230104/7f44c158/attachment.html>