Article Extraction API

Allows developers to extract the main text of an HTML article. This includes the main text and html bodies, the author, any embeddable media such as youtube and twitter cards, the RSS feeds and other social feeds like Facebook and Twitter timelines or Youtube channels, and other meaningful pieces of data.

Request

To use the Article Extraction API, perform a HTTP GET request on the following endpoint:

https://lexper.p.rapidapi.com/v1.1/extract

Provide the following arguments:

Argument Description
url URL of article to be extracted.
Optional arguments
js indicates whether to execute JavaScript or not, set to auto to let the extractor decide whether to run JavaScript or not.
text indicates whether API should return extracted text
html indicates whether API should extract html
media indicates whether API should extract media
feeds indicates whether API should extract RSS feeds
images indicates whether API should extract all images present in HTML.
author indicates whether API should extract article's author
pub_date indicates whether API should extract article's publish date
partial number of characters or percentage of text (if percent sign is present) of text/html to be returned
strip_tags indicates which tags to strip from the extracted HTML. Expects a comma separated list of tag names.
timeout request timeout in seconds.
js_timeout when JavaScript is enabled, indicates how many seconds the API should wait for the JS interpreter before starting the extraction.
image_analysis indicates whether API should analyse images for minimum width and height (see parameters min_image_width and min_image_height for more details).
min_image_width minimum width of the images kept in the HTML (if image_analysis is false this parameter has no effect).
min_image_height minimum height of the images kept in the HTML (if image_analysis is false this parameter has no effect).
image_timeout image grabbing timeout in seconds.
use_proxy indicates whether request should be relayed via proxy. Some sites might block requests that originate in data centers, so it is sometimes a good idea to circumvent their check by using a proxy. Please note that this feature is only available to paying users. Defaults to 1 (if available).
pagination extract and concatenate multiple-page articles, enabled by default.
pagination_max_pages number of pages to extract when pagination is enabled.
Custom Headers
LXF-headerName indicates which headers to send to target URL. This can be useful when article is behind a pay wall for example.
curl -i -H 'LXF-Referer: lexper.com' -H 'LXF-User-Agent: Lexper API' -X GET https://lexper.p.mashape.com/v1.1/extract?url=https://medium.com/personal-growth/how-to-be-yourself-2221085391a3

Example Response

{
    "article": {
        "text": "I began learning German at the age of 13, and I’m still trying to explain to myself why it was love at first sound. The answer must surely be: the excellence of my teacher. At an English public school not famed for its cultural generosity, Mr King was that rare thing: a kindly and intelligent man who, in the thick of the second world war, determinedly loved the Germany that he knew was still there somewhere.\nRather than join the chorus of anti-German propaganda, he preferred, doggedly, to inspire his little class with the beauty of the language, and of its literature and culture. One day, he used to say, the real Germany will come back. And he was right. Because now it has.\nWhy was it love at first sound for me? Well, in those days not many language teachers played gramophone records to their class, but Mr King did. They were old and very precious to him and us, and he kept them in brown paper bags in a satchel that he put in his bicycle basket when he rode to school.\nWhat did they contain, these precious records? The voices of classical German actors, reading romantic German poetry. The records were a bit cracked, but that was part of their beauty. In my memory, they remain cracked to this day:\nDu bist wie eine Blume – CRACK – So hold und schön und... – CRACK (Heinrich Heine)\nBei Nacht im Dorf der Wächter rief… – CRACK (Eduard Mörike’s Elfenlied)\nAnd I loved them. I learned to imitate, then recite them, crack and all. And I discovered that the language fitted me. It fitted my tongue. It pleased my Nordic ear.\nI also loved the idea that these poems and this language that I was learning were mine and no one else’s, because German wasn’t a popular subject and very few of my schoolmates knew a word of it beyond the Achtung! and Hände hoch! that they learned from propaganda war movies.\nBut thanks to Mr King, I knew better. And when I decided I couldn’t stand my English public school for one more day, it was the German language that provided me with my bolt-hole. The year was 1948. I couldn’t go to Germany, so I went to Switzerland and at 16 enrolled myself at Bern university.\nAnd in Switzerland, instead of Mr King, I had another admirable teacher in Frau Karsten, a stern north German lady with grey hair in a ponytail, and she, like Mr King, rode a bicycle, sitting very upright with her grey hair bobbing along behind her.\nSo it’s no wonder that when later I went into the army for my national service, I was posted to Austria. Or that after the army I went on to study German at Oxford. And so to Eton, to teach it.\nYou can have a lot of fun with the German language, as we all know. You can tease it, play with it, send it up. You can invent huge words of your own – but real words all the same, just for the hell of it. Google gave me Donaudamp-fschiffsfahrtsgessellschaftskapitän.\nYou’ve probably heard the Mark Twain gag: “Some German words are so long they have a perspective.” You can make up crazy adjectives like “my-recently-by-my-parents-thrown- out-of- the-window PlayStation”. And when you’re tired of floundering with nouns and participles strung together in a compound, you can turn for relief to the pristine poems of a Hölderlin, or a Goethe, or a Heine, and remind yourself that the German language can attain heights of simplicity and beauty that make it, for many of us, a language of the gods.\nAnd for all its pretending, the German language loves the simple power of monosyllables.\nThe decision to learn a foreign language is to me an act of friendship. It is indeed a holding out of the hand. It’s not just a route to negotiation. It’s also to get to know you better, to draw closer to you and your culture, your social manners and your way of thinking. And the decision to teach a foreign language is an act of commitment, generosity and mediation.\nIt’s a promise to educate – yes – and to equip. But also to awaken; to kindle a flame that you hope will never go out; to guide your pupils towards insights, ideas and revelations that they would never have arrived at without your dedication, patience and skill.\nTo quote Charlemagne: “To have another language is to possess a second soul.” He might have added that to teach another language is to implant a second soul.\nOf course, the very business of reconciling these two souls at any serious level requires considerable mental agility. It compels us to be precise, to confront meaning, to think rationally and creatively and never to be satisfied until we’ve hit the equivalent word, or – which also happens – until we’ve recognised that there isn’t one, so hunt for a phrase or circumlocution that does the job.\nNo wonder then that the most conscientious editors of my novels are not those for whom English is their first language, but the foreign translators who bring their relentless eye to the tautological phrase or factual inaccuracy – of which there are far too many. My German translator is particularly infuriating.\nIn the extraordinary period we are living through – may it be short-lived – it’s impossible not to marvel at every contradictory or unintelligible utterance issuing from across the Atlantic. And in marvelling, we come face-to-face with the uses and abuses of language itself.\nClear language – lucid, rational language – to a man at war with both truth and reason, is an existential threat. Clear language to such a man is a direct assault on his obfuscations, contradictions and lies. To him, it is the voice of the enemy. To him, it is fake news. Because he knows, if only intuitively, what we know to our cost: that without clear language, there is no standard of truth.\nAnd that’s what language means to a linguist. Those who teach language, those who cherish its accuracy and meaning and beauty, are the custodians of truth in a dangerous age.\nAnd if they teach German – and teach it in this my beleaguered country – they are quite particularly to be prized, all the more so because they are an endangered species. Every time I hear a British politician utter the fatal words, “Let me be very clear”, these days I reach for my revolver.\nBy teaching German, by spreading understanding of German culture and life, today’s honorands and their colleagues will be helping to balance the European argument, to make it decent, to keep it civilised.\nThey will be speaking above all to this country’s most precious asset: its enlightened young, who – Brexit or no Brexit – see Europe as their natural home, Germany as their natural partner, and shared language as their natural bond.\nAfter studying in Switzerland and national service, John le Carré (aka David Cornwell) was posted to the British Embassy in Bonn as a Second Secretary, and then went on to Hamburg as British Consul. This text was originally part of a speech given by le Carré at an award ceremony at the German Embassy for German teachers on 12 June.",
        "html": "<p><span><span>I<\/span><\/span> began learning German at the age of 13, and I’m still trying to explain to myself why it was love at first sound. The answer must surely be: the excellence of my teacher. At an English public school not famed for its cultural generosity, Mr King was that rare thing: a kindly and intelligent man who, in the thick of the second world war, determinedly loved the Germany that he knew was still there somewhere.<\/p>\n<p>Rather than join the chorus of anti-German propaganda, he preferred, doggedly, to inspire his little class with the beauty of the language, and of its literature and culture. One day, he used to say, the real Germany will come back. And he was right. Because now it has.<\/p>\n<p>Why was it love at first sound for me? Well, in those days not many language teachers played gramophone records to their class, but Mr King did. They were old and very precious to him and us, and he kept them in brown paper bags in a satchel that he put in his bicycle basket when he rode to school.<\/p>\n<p>What did they contain, these precious records? The voices of classical German actors, reading romantic German poetry. The records were a bit cracked, but that was part of their beauty. In my memory, they remain cracked to this day:<\/p>\n<p><em>Du bist wie eine Blume<\/em> – CRACK – <em>So hold und schön und...<\/em> – CRACK (Heinrich Heine)<\/p>\n<p><em>Bei Nacht im Dorf der Wächter rief…<\/em> – CRACK (Eduard Mörike’s <em>Elfenlied<\/em>)<\/p>\n<p>And I loved them. I learned to imitate, then recite them, crack and all. And I discovered that the language fitted me. It fitted my tongue. It pleased my Nordic ear.<\/p>\n<p>I also loved the idea that these poems and this language that I was learning were mine and no one else’s, because German wasn’t a popular subject and very few of my schoolmates knew a word of it beyond the <em>Achtung!<\/em> and <em>Hände hoch!<\/em> that they learned from propaganda war movies.<\/p>\n<p>But thanks to Mr King, I knew better. And when I decided I couldn’t stand my English public school for one more day, it was the German language that provided me with my bolt-hole. The year was 1948. I couldn’t go to Germany, so I went to Switzerland and at 16 enrolled myself at Bern university.<\/p>\n<p>And in Switzerland, instead of Mr King, I had another admirable teacher in Frau Karsten, a stern north German lady with grey hair in a ponytail, and she, like Mr King, rode a bicycle, sitting very upright with her grey hair bobbing along behind her.<\/p>\n<p>So it’s no wonder that when later I went into the army for my national service, I was posted to Austria. Or that after the army I went on to study German at Oxford. And so to Eton, to teach it.<\/p>\n<p>You can have a lot of fun with the German language, as we all know. You can tease it, play with it, send it up. You can invent huge words of your own – but real words all the same, just for the hell of it. Google gave me <em>Donaudamp-fschiffsfahrtsgessellschaftskapitän<\/em>.<\/p>\n<p>You’ve probably heard the Mark Twain gag: “Some German words are so long they have a perspective.” You can make up crazy adjectives like “my-recently-by-my-parents-thrown- out-of- the-window PlayStation”. And when you’re tired of floundering with nouns and participles strung together in a compound, you can turn for relief to the pristine poems of a Hölderlin, or a Goethe, or a Heine, and remind yourself that the German language can attain heights of simplicity and beauty that make it, for many of us, a language of the gods.<\/p>\n<p>And for all its pretending, the German language loves the simple power of monosyllables.<\/p>\n<p>The decision to learn a foreign language is to me an act of friendship. It is indeed a holding out of the hand. It’s not just a route to negotiation. It’s also to get to know you better, to draw closer to you and your culture, your social manners and your way of thinking. And the decision to teach a foreign language is an act of commitment, generosity and mediation.<\/p>\n<p>It’s a promise to educate – yes – and to equip. But also to awaken; to kindle a flame that you hope will never go out; to guide your pupils towards insights, ideas and revelations that they would never have arrived at without your dedication, patience and skill.<\/p>\n<p>To quote Charlemagne: “To have another language is to possess a second soul.” He might have added that to teach another language is to implant a second soul.<\/p>\n<p>Of course, the very business of reconciling these two souls at any serious level requires considerable mental agility. It compels us to be precise, to confront meaning, to think rationally and creatively and never to be satisfied until we’ve hit the equivalent word, or – which also happens – until we’ve recognised that there isn’t one, so hunt for a phrase or circumlocution that does the job.<\/p>\n<p>No wonder then that the most conscientious editors of my novels are not those for whom English is their first language, but the foreign translators who bring their relentless eye to the tautological phrase or factual inaccuracy – of which there are far too many. My German translator is particularly infuriating.<\/p>\n<p>In the extraordinary period we are living through – may it be short-lived – it’s impossible not to marvel at every contradictory or unintelligible utterance issuing from across the Atlantic. And in marvelling, we come face-to-face with the uses and abuses of language itself.<\/p>\n<p>Clear language – lucid, rational language – to a man at war with both truth and reason, is an existential threat. Clear language to such a man is a direct assault on his obfuscations, contradictions and lies. To him, it is the voice of the enemy. To him, it is fake news. Because he knows, if only intuitively, what we know to our cost: that without clear language, there is no standard of truth.<\/p>\n<p>And that’s what language means to a linguist. Those who teach language, those who cherish its accuracy and meaning and beauty, are the custodians of truth in a dangerous age.<\/p>\n<p>And if they teach German – and teach it in this my beleaguered country – they are quite particularly to be prized, all the more so because they are an endangered species. Every time I hear a British politician utter the fatal words, “Let me be very clear”, these days I reach for my revolver.<\/p>\n<p>By teaching German, by spreading understanding of German culture and life, today’s honorands and their colleagues will be helping to balance the European argument, to make it decent, to keep it civilised.<\/p>\n<p>They will be speaking above all to this country’s most precious asset: its enlightened young, who – Brexit or no Brexit – see Europe as their natural home, Germany as their natural partner, and shared language as their natural bond.<\/p>\n<p><em>After studying in Switzerland and national service, John le Carré (aka David Cornwell) was posted to the British Embassy in Bonn as a Second Secretary, and then went on to Hamburg as British Consul. This text was originally part of a speech given by le Carré at an award ceremony at the German Embassy for German teachers on 12 June.<\/em><\/p>",
        "media": [],
        "images": [],
        "author": "John le Carré",
        "feeds": [],
        "pub_date": "2017-07-01 23:05:12",
        "url": "https:\/\/www.theguardian.com\/education\/2017\/jul\/02\/why-we-should-learn-german-john-le-carre",
        "canonical_url": "https:\/\/www.theguardian.com\/education\/2017\/jul\/02\/why-we-should-learn-german-john-le-carre",
        "title": "Why we should learn German | John le Carré",
        "language": "en",
        "image": "https:\/\/i.guim.co.uk\/img\/media\/f19eff6f7e1751d88b38e725cfbe6687084d5f64\/0_235_9010_5405\/master\/9010.jpg?width=1200&height=630&quality=85&auto=format&fit=crop&overlay-align=bottom%2Cleft&overlay-width=100p&overlay-base64=L2ltZy9zdGF0aWMvb3ZlcmxheXMvdG8tb3BpbmlvbnMucG5n&enable=upscale&s=5bfc15734393dfa1ea74afd347789c47",
        "summary": "To help make the European debate decent and civilised, it is now more important than ever to value the skills of the linguist",
        "modified_date": "2017-12-02 03:00:56",
        "site_name": "the Guardian",
        "favicon": "https:\/\/assets.guim.co.uk\/images\/favicons\/46bd2faa1ab438684a6d4528a655a8bd\/32x32.ico",
        "encoding": "utf-8",
        "pages": [
            "https:\/\/www.theguardian.com\/education\/2017\/jul\/02\/why-we-should-learn-german-john-le-carre"
        ]
    },
    "time": 6.037055015563965,
    "js": false,
    "proxy": false,
    "pagination": false
}

Try out Ujeebu

Enter a URL in the box below