[SOLVED] Translating %-introduced hex characters into ASCII

RandomTroll · 05-17-2020, 05:12 PM

I want to extract information from files that render a lot
of non-alphanumeric characters in hex, for example '%3D' for
'=', '%2B' for '+'. Is there a name for this practice? Is
there already a program that translates them into ASCII, which
would save me the trouble of writing one? Here's a sample line:

Quote:

<a href="#" data-db-target-for="7053458b-d78a-4662-8e29-53bc26fbb44e" data-db-switch="" aria-haspopup="true" aria-controls="7053458b-d78a-4662-8e29-53bc26fbb44e_Pop" role="button" id="7053458b-d78a-4662-8e29-53bc26fbb44e_Ctrl" data-slide-target="#7053458b-d78a-4662-8e29-53bc26fbb44e_Pop" class="w-slide__btn visibleOverflow">Tools</a><div data-db-target-of="7053458b-d78a-4662-8e29-53bc26fbb44e" aria-labelledby="7053458b-d78a-4662-8e29-53bc26fbb44e_Ctrl" role="menu" id="7053458b-d78a-4662-8e29-53bc26fbb44e_Pop"><ul class="rlist w-slide--list"><li role="presentation" class="article-tool"><a href="/personalize/addFavoritePublication?doi=10.7326%2FL19-0550" role="menuitem">Add to favorites</a></li><li role="presentation" class="article-tool"><a href="/action/showCitFormats?doi=10.7326%2FL19-0550" role="menuitem">Download Citations</a></li><li role="presentation" class="article-tool"><a href="/action/addCitationAlert?doi=10.7326%2FL19-0550" role="menuitem">Track Citations</a></li><li role="presentation" class="article-tool"><a href="/servlet/linkout?type=rightslink&url=startPage%3D634%26pageCount%3D3%26author%3DAlan%2BJ.%2BHunter%26orde rBeanReset%3Dtrue%26imprint%3DAmerican%2BCollege%2Bof%2BPhysicians%26volumeNum%3D172%26issueNum%3D9% 26contentID%3D10.7326%252FL19-0550%26title%3DUnilateral%2BFacial%2BParalysis%2BDuring%2Ban%2BAirline%2BFlight%26numPages%3D3%26pa% 3D%26issn%3D0003-4819%26publisherName%3DACP%26publication%3D0003-4819%26rpt%3Dn%26endPage%3D636%26publicationDate%3D05%252F05%252F2020" role="menuitem" target="popup" onclick="window.open("/servlet/linkout?type=rightslink&url=startPage%3D634%26pageCount%3D3%26author%3DAlan%2BJ.%2BHunter%26orde rBeanReset%3Dtrue%26imprint%3DAmerican%2BCollege%2Bof%2BPhysicians%26volumeNum%3D172%26issueNum%3D9% 26contentID%3D10.7326%252FL19-0550%26title%3DUnilateral%2BFacial%2BParalysis%2BDuring%2Ban%2BAirline%2BFlight%26numPages%3D3%26pa% 3D%26issn%3D0003-4819%26publisherName%3DACP%26publication%3D0003-4819%26rpt%3Dn%26endPage%3D636%26publicationDate%3D05%252F05%252F2020","popup"," width=750,height=650&quot

; return false;">Permissions</a></li></ul></div>

michaelk · 05-17-2020, 05:34 PM

URLs containing reserved characters are encoded to % hex characters.

https://en.wikipedia.org/wiki/Percent-encoding

There are various ways of decoding hex characters using whatever language / tool you want... bash, sed, php, python etc you prefer.

syg00 · 05-17-2020, 06:07 PM

Browsers, even the CLI ones, know how to interpret that stuff.

RandomTroll · 05-19-2020, 01:14 AM

Quote:

Originally Posted by michaelk

There are various ways of decoding hex characters using whatever language / tool you want... bash, sed, php, python etc you prefer.

I had already written a program to do this but had forgotten about it.

Quote:

Originally Posted by syg00

Browsers, even the CLI ones, know how to interpret that stuff.

I want to extract information programatically.

Turbocapitalist · 05-19-2020, 01:30 AM

It's an encoding and decoding process not extraction. See RFC 3986.

Code:

perl -MURI::Escape -n -e 'print uri_escape($_);'

See "man URI::Escape"

RandomTroll · 05-20-2020, 12:11 AM

Quote:

Originally Posted by Turbocapitalist

It's an encoding and decoding process not extraction.

I started out wanting to extract data from an HTML page that uses this encoding protocol. I had to decode to extract; decoding is a subsidiary process.