academic.bancey.comDr. Simon Bance – Scientific computer simulation, magnetic materials and devices

Dr. Simon Bance Scientific computer simulation, magnetic materials and devices Blog Research Publications Twitter ResearchGate LinkedIn About Menu Blog Research Publications Twitter ResearchGate LinkedIn About Search for: May 16 2019 May 16, 2019 May 16, 2019 by Simon Fixing a USB Flash drive that has been “corrupted” by balenaEtcher. Posted in Uncategorized Recently I used balenaEtcher to create a bootable Linux drive. It is a nice Windows program that simplifies the process of creating Linux disks and is recommended by some Linux distro vendors. After the program successfully writes a bootable Linux USB drive, the drive no longer appears correctly in Windows. That’s because a bootable USB contains certain drive partitions with filesystem types that are not visible to the Windows operating system. This is not really a problem until you want to remove Linux from the USB drive and start using it, once again, as a normal drive for transferring files etc. Windows is unable to see the partitions correctly and is therefore unable to remove them, or format the drive correctly. The only way I found to “repair” my pen drive was by using a computer that was already running Linux, in my case Ubuntu. I used gparted, which comes already installed in Ubuntu, to remove all partitions on the USB drive, which for me was mounted at /sdb1/ You should take care that you are modifying the correct drive location! Once I deleted any existing partitions I created a new partition from the unallocated space, with an NTFS filesystem that is visible in Windows as well as Linux. For good measure I thoroughly formatted the drive a second time using the option in the File Explorer menu system. Now when I check my drive in Windows it appears as it should; one large partitions with (almost) the whole capacity of the USB available. A lot of people have written on the balenaEtcher forums about this problem, blaming the software for destroying their USB drives. This is not the case as the drives can be fixed, but it would be nice if balena addressed this issue by providing some kind of tool for restoring a drive to its original state. Tags: LINUX Software Ubuntu Nov 30 2018 November 30, 2018 December 3, 2018 No Comment by Simon Adding OCR layers to your Zotero library PDF items for Metadata extraction and indexing Posted in Software Zotero is a cross-platform literature manager that is able to sync to a remote server and across multiple user devices. There are many alternatives available, each with strengths and weaknesses, but I am currently using Zotero to manage my literature because it is free and works with WebDAV for additional free storage. In this article I will describe why optical character recognition (OCR) is important for Zotero and suggest a way to add OCR to existing items in a Zotero library. However, the method actually works for any collection of PDF files on your computer! The reason for OCR in Zotero Zotero has a nice “Retrieve Metadata for PDF” feature that automatically scans a PDF file for metadata and then uses it to search for matching bibliography information from Google Scholar. The PDF is then nested under a parent item that is (usually) properly indexed in the internal Zotero SQLite database. In this case Zotero found matches for most of the items. The one with the red cross appears not to be a journal article or book, but some other random (non-public) document that at some time was imported into my library. However, if the PDF does not contain an OCR layer this feature does not work. This is often the case for older journal articles, or PDFs that were scanned from a hard copy. If you manage a large literature library then you might have many non-OCR files in there, which are not properly indexed. Manually creating a parent item for each of them is laborious. The only practical approach is to add an OCR layer to the PDF files. Example errors when Zotero is unable to find an OCR layer in a PDF document during attempted metadata retrieval. Adding OCR to PDF files There are a number of commercial, free and open source options for adding OCR to PDF files. Most famous of these is the Adobe Acrobat reader, which at the time of writing requires a monthly subscription to an “Edit” feature extension to unlock the OCR capabilities. If you have this available to you, please go ahead and use it. If you prefer a free option there are a few available, but I had most success with ocrmypdf , written by James R. Barlow and release under the GNU GPLv3 license. The following steps should help you get started with ocrmypdf and use it to fix those annoying OCR problems in Zotero. Installing ocrmypdf Linux I am using Ubuntu 18.0.4.1LTS. Before using apt-get to install ocrmypdf, it was necessary to allow additional software to be installed Ubuntu software repository options. Then you should be able to do: sudo apt-get install ocrmypdf For more installation information please visit the project page . General usage On the command line terminal you can simply provide ocrmypdf with an input PDF file and the desired output file. ocrmypdf input.pdf output.pdf If successful, this creates a new file called output.pdf, which is a modified version of the original. The new file should hopefully contain an OCR text layer! Usage with Zotero My aim here is to describe a method for parsing through a large Zotero library, checking for files without an OCR layer and then adding one on the fly. We will eventually write abash script to control this, but first I will explain how individual steps in the script work. One-liner for a single file In this crude example I have created a new folder called /home/simon/Zotero/ocr/. The Zotero storage folder is in /home/simon/Zotero/storage, or simply ../storage/ The following allows you to find a file using a search string, for example here the filename ends with ” – kittel.pdf.pdf”. I am assuming there is only one file that matches this search string! INPUT=`find ../storage/ -name "* - kittel.pdf.pdf"`; ocrmypdf "$INPUT" output.pdf You can then check the output.pdf manually then replace the original. If you are feeling really brave, and wish to do this on the fly, without checking the new PDF file first, you can automatically replace the input file directly in the Zotero storage folder. INPUT=`find ../storage/ -name "* - kittel.pdf.pdf"`; ocrmypdf "$INPUT" output.pdf; mv output.pdf "$INPUT" I did not have any issues with this approach as ocrmypdf doesn’t seem to be destructive, but care should be taken when automatically replacing or deleting files! In the final script below I take a few rudimentary precautions in that sense! Check if a file has OCR We are going to approach this method on the bash command line. To that end we can use pdffonts to check whether a PDF document contains OCR. pdffonts checks if the file contains embedded fonts. Any PDF without OCR text will contain zero embedded fonts, while a file with an OCR layer will have 1 or more embedded fonts. For example, this file does have an OCR layer: $ pdffonts output2.pdf name type encoding emb sub uni object ID ------------------------------------ ----------------- ---------------- --- --- --- --------- JKMERI+GlyphLessFont CID TrueType Identity-H yes yes yes 9 0 QMAGQI+GlyphLessFont CID TrueType Identity-H yes yes yes 20 0 MEMQSK+GlyphLessFont CID TrueType Identity-H yes yes yes 38 0 So does this one: pdffonts Unknown\ -\ Unknown\ -\ Chapter\ 28\ –\ Magnetic\ Fields\ Sources\ Goals\ for\ Chapter\ 28.pdf name type encoding emb sub uni object ID ------------------------------------ ----------------- ---------------- --- --- --- --------- ABCDEE+Calibri TrueType WinAnsi yes yes no 5 0 ABCDEE+Calibri CID TrueType Identity-H yes yes yes 7 0 Times New Roman CID TrueType Identity-H yes no yes 14 0 Times New Roman TrueType WinAnsi no no no 19 0 Times New Roman,Bold TrueType WinAnsi no no no 30 0 Symbol CID TrueType Identity-H yes no yes 32 0 Times New Roman,Italic TrueType WinAnsi no no no 55 0 ABCD...