Reply
User
Posts: 6
Registered: ‎10-30-2013
0

automatically update references

I have been an endnote user for quite a while and I thus have a library with Pubmed entries that are quite dated. It would be very useful to update all newly added fields automatically. Currently, this can be done by selecting all refs and then selecting update. However, for each reference I am asked to first select "empty fields only" and then "save update". For a library with over 2000 entries this is prohibitive. In fact, I got tired of it after 10!

So an option where the fields to be updated is selected once and then hit go would be greatly appreciated. It seems this should be trival to program but would be amazingly useful for many researchers now having to add new fields for the NIH grant proposals.

 

THANKS!!

Moderator
Posts: 292
Registered: ‎03-27-2008
0

Re: automatically update references

[ Edited ]

Hello, felixschweizer:

 

I've spoken with a number of users here in Support who have asked for a feature like this. I can't speak for the product managers, but what I always point out to those customers that while that feature certainly would be helpful, it could also be highly dangerous.

 

At the moment, you are prompted individually for a very very good reason. Let's say you've been working with your references and adding extensive notes into a custom field, such as notes or research notes. This is a pretty common workflow for people, and sometimes they invest large chunks of time into adding this custom information.

 

If we had an automatic update wherein it did not compare the references, it could possibly -- without notification -- wipe out the custom information you added. That is why, right now, it asks you very clearly and carefully for each individual reference whether it should overwrite fields or only update the empty fields... the first option could have the potential of removing some of your custom information, and the second could miss valuable new information. And one record may be better with one solution, and another the other.

 

I still think there are likely ways EndNote could do a better job with some of these things than it does right now, so this is certainly a valuable thread to have here in the Suggestions forum. However, I wanted to make sure it is clear why it does what it does right now, as tedious as it may seem -- keeping your data safe is our paramount concern.

 

 

Gillian Neff
Supervisor, Customer Service
Customer Care

Clarivate Analytics
Formerly the IP & Science business of Thomson Reuters

clarivate.com
ipscience.tr.com
Contributor
Posts: 42
Registered: ‎12-17-2011
0

Re: automatically update references

This is an annoyance, but here is a solution of sorts.  Usually once the publisher updates the record with keywords and such, the record doesn't change further.  It CAN happen of course, but not often.

 

If I grab a newish article, perhaps from the "Early Online" section of a journal, there will be a Pubmed record that will later get updated.  My work around was to create a custom group called "Update" that contains a subset of my database for any reference that is missing the accession number, volume/issue number, or keywords.  If the journal or item is not going to be indexed in Pubmed (like unpublished papers I manage to obtain), I add the words "Not indexed in Pubmed" into the accession number field and those are excluded from my custom group.

 

Presently I have about 2,800 articles in my Endnote, but only 25 or so in that custom group.  When I need to update I only have to scan the 25 that are missing full information and as those records get updated will full information they drop our of the group automatically.  Some publishers update with a month or two, some take up to a year (certain IngentaConnect publications are the worst).  Finally, in order to avoid having to confirm the updates manually I wrote an Autohotkey macro that does it for me. 

 

One I have the macro loaded, I select all the records in my "Update" group and then click on the "Find Reference Updates" and Autohotkey will do the mouse clicks necessary to confirm any updates available.  With only 25 records  I can usually scan all my incomplete references while I refill my coffee, and I only run the macro once every few weeks.  Autohotkey is a free program (Google it) but it takes a bit of practice to get it to work.  The mouse click coordinates positions are specific for YOUR monitor dimensions and screen resolution, and the sleep timer depends on how fast your computer is (you might need to make it longer if you have a slow computer).  Also note that Autohotkey "reads" the window titles so Endnote must be on top while this is running (i.e. you can't start the macro and then go web browsing while it runs).  With those caveats, here is the macro in all its glory

 

settitlematchmode, 1
start:
WinWaitActive Review
click 430,130
sleep 100
Click 615,415
goto start

 

The problem with having Endnote do an automatic update, aside from overwriting custom data as was mentioned, is that Endnote / ThompsonReuters doesn't control the Pubmed database.  If you have over 2,000 entries then Endnote would have to connect to Pubmed 2,000 times and request each record one by one every time you wanted to update.  Even with a fast connection that would take many hours to scan your full database, as it did mine, which is why I created the custom group to limit the effort to just the records that need attention. 

 

Note that my macro is fully automatic, so it will overwrite any custom notes you have added so use it at your own risk.  It works for me, but a wise person would test it out on a COPY of their database.

Mentor
Posts: 6,819
Registered: ‎04-10-2008
0

Re: automatically update references

Warning:

 

One thing to be careful of with in reference updates and find full text calls to Pubmed, is that Pubmed may interpret high numbers and quick repeat requests as Denial of Service attacks and block your IP address, so running something like this on 2000 records is a bit dangerous anyway.  

(long time Endnote user)
Frequent Contributor
Posts: 58
Registered: ‎03-15-2012
0

Re: automatically update references

[ Edited ]

Hodag1, your script sounds like a good interim solution.  Can I just enter the text into AHK as you have it?  Is that the exact script? Are there headers?

Thomas Zucker-Scharff
Systems Analyst
Frequent Contributor
Posts: 58
Registered: ‎03-15-2012
0

Re: automatically update references

[ Edited ]

Tried script on my windows 7 box using EndNote x7.02 and it doesn't do anything.  Are there other lines that need to be there?  The script is currently just the default with the commented lines left in and the uncommented one replaced with your script. (I used the zip archive to install)

 

; IMPORTANT INFO ABOUT GETTING STARTED: Lines that start with a
; semicolon, such as this one, are comments.  They are not executed.

; This script has a special filename and path because it is automatically
; launched when you run the program directly.  Also, any text file whose
; name ends in .ahk is associated with the program, which means that it
; can be launched simply by double-clicking it.  You can have as many .ahk
; files as you want, located in any folder.  You can also run more than
; one .ahk file simultaneously and each will get its own tray icon.

; SAMPLE HOTKEYS: Below are two sample hotkeys.  The first is Win+Z and it
; launches a web site in the default browser.  The second is Control+Alt+N
; and it launches a new Notepad window (or activates an existing one).  To
; try out these hotkeys, run AutoHotkey again, which will load this file.


settitlematchmode, 1
start:
WinWaitActive Review
click 430,130
sleep 100
click 615,415
goto start


; Note: From now on whenever you run AutoHotkey directly, this script
; will be loaded.  So feel free to customize it to suit your needs.

; Please read the QUICK-START TUTORIAL near the top of the help file.
; It explains how to perform common automation tasks such as sending
; keystrokes and mouse clicks.  It also explains more about hotkeys.

Thomas Zucker-Scharff
Systems Analyst
Contributor
Posts: 42
Registered: ‎12-17-2011
0

Re: automatically update references

No, that is the entire script.  I have added one additional line since to make it run more slowly, and was more verbose on the WinWaitActive command but that really shouldn't matter:

 

Settitlematchmode, 2
start:
WinWaitActive Review Available Updates
click 430,130
sleep 250
Click 615,415
Sleep 1000
Goto start

 

What is likely your problem is that either you did not update the mouse click coordinates for YOUR screen, you did not update the sleep parameter for YOUR computer, or you made another error.

 

Play with the AHK "window spy" and make sure that the mouse coordinates specified in the script match where the boxes appear on your screen.  Personally, I have a Lenovo T420 laptop with a 14" display.  If you have any other size display then your mouse will be in the wrong spot and you will need to update the coordinates.  Ditto if you sometimes run in a window instead of full screen, the coordinates will change on you as you resize the window.  I only use EN in full screen so I don't have that issue, but I did have to tweak the macro when I bought this computer and retired my older one with a 12" display.

 

Second issue is that the script runs too fast for your computer.  If yours is slower than mine, then you may need to have more "Sleep" units between commands to give the computer a chance to catch up to the script.  This is something you do by trial and error, just add a Sleep 5000 command for example to give it a bit of pause (each sleep unit is one millisecond, so 5000= 5 seconds) and if that is taking too long reduce it to Sleep 3000 or 1000 as necessary.  You have to customize it to the speed of your computer processor and your Internet link.

 

Finally, are you loading the AHK macro into memory, highlighting the records you need to update, and then using the "Find Reference Updates" on the set?  Just try to get it working on two records, and if that works then you are good to go with more (although you may need to tweak the "Sleep" a bit more).  Also, EN must be the window on top.  You can't start the macro and then go Internet surfing; EN must be the program showing in the main window and it must be on top.

 

As for the user that cautioned running too many requests at once, they are correct.  However, you can do small subsets (like 25 records at a time) until your database is fully ready, or you can insert a one minute delay into the macro with Sleep 60000 so it only hits PubMed once a minute.  Leave it running overnight and you can do several hundred records a night for a few nights.  Personally, I do record updates of 25-50 all the time without any issues, but I am the only person that uses this IP address to contact PubMed.  I am not sure how it would look to Pubmed if, for example, you were on a shared IP from a university network with lots of students sharing a single routable address from behind a router and you all used PubMed at the same time.

 

 

Frequent Contributor
Posts: 58
Registered: ‎03-15-2012
0

Re: automatically update references

so does that script need to be part of the default autohotkey script?  I was trying to make a new one.

Thomas Zucker-Scharff
Systems Analyst
Frequent Contributor
Posts: 58
Registered: ‎03-15-2012
0

Re: automatically update references

With a little help from colleagues I created this script to auto update my records.  The procedure is as follows:

 

Download AutoHotKey

Install AHK

comment all lines out of the default script (if you aren't going to make a new one)

add the following lines in where the script was

 

Loop
{
  SetTitleMatchMode,1 ; title must begin with string
  IfWinExist,Review Available Updates
  {
    WinActivate
    Send {Enter}
    Sleep,100
    Send {Tab}
    Sleep,100
    Send {Enter}
    Sleep,3000 ; wait 3 seconds (parameter is milliseconds)
  }
}

 

So the default script would look like this (feel free to copy and paste any of this post):

 


; IMPORTANT INFO ABOUT GETTING STARTED: Lines that start with a
; semicolon, such as this one, are comments.  They are not executed.

; This script has a special filename and path because it is automatically
; launched when you run the program directly.  Also, any text file whose
; name ends in .ahk is associated with the program, which means that it
; can be launched simply by double-clicking it.  You can have as many .ahk
; files as you want, located in any folder.  You can also run more than
; one .ahk file simultaneously and each will get its own tray icon.

; SAMPLE HOTKEYS: Below are two sample hotkeys.  The first is Win+Z and it
; launches a web site in the default browser.  The second is Control+Alt+N
; and it launches a new Notepad window (or activates an existing one).  To
; try out these hotkeys, run AutoHotkey again, which will load this file.

;#z::Run www.autohotkey.com

Loop
{
  SetTitleMatchMode,1 ; title must begin with string
  IfWinExist,Review Available Updates
  {
    WinActivate
    Send {Enter}
    Sleep,100
    Send {Tab}
    Sleep,100
    Send {Enter}
    Sleep,3000 ; wait 3 seconds (parameter is milliseconds)
  }
}


;^!n::
;IfWinExist Untitled - Notepad
; WinActivate
;else
; Run Notepad
;return
;
;
; Note: From now on whenever you run AutoHotkey directly, this script
; will be loaded.  So feel free to customize it to suit your needs.

; Please read the QUICK-START TUTORIAL near the top of the help file.
; It explains how to perform common automation tasks such as sending
; keystrokes and mouse clicks.  It also explains more about hotkeys.

 

 

You can change things if you want to only update empty fields (instead of the Send {Return} command first precede it with a Send {Tab}), or if you wish you can set a longer period at the end (instead of sleep,3000 - 3 seconds, you could set it for 120000 - 2 minutes)

Thomas Zucker-Scharff
Systems Analyst
Frequent Contributor
Posts: 58
Registered: ‎03-15-2012
0

Re: automatically update references

I tried the Send {tab} option out and it doesn't work.  If there isn't the option to update only the empty fields it skips that and acts differently.  So you would have to query if the button was active first.

Thomas Zucker-Scharff
Systems Analyst