[grc] How to automate downloading Making Contact?

Sun Jul 26 09:51:38 PDT 2020

Hi, everyone.  Thanks for your responses about downloading Making Contact.

Paul Bame: The doc for curl (and wget) says that robots.txt is ignored, and
that seems to be the case in my testing.

David Klann: I want to automate the download process, which means the
program file name has to be something that can be predicted.  The Audioport
links don't look like that's the case.  We do have a membership with
Audioport and once I get our credentials I'll explore what is available
there.

Carl Etnier: Thanks, that sounds the most promising.  I'll check to see who
at our station has a Pacifica userid, and track down your presentation.

I did some more research, and it appears the RadioProject website is using
the Incapsula product to control access to the files.  Incapsula inserts a
script in the web link and requires you to go through that to get to the
thing you want.  There are hack=y ways around the script, but none that
work reliably.

Regards,
Mike

On Sat, Jul 25, 2020 at 5:19 AM Paul Bame <pbame at prometheusradio.org> wrote:

> There should be a curl option to ignore robots.txt
>
>
>
>
>
>
>
> Get BlueMail for Android <http://www.bluemail.me/r?b=15860>
> On Jul 25, 2020, at 08:16, Mike Carroll via grc <grc at maillist.peak.org>
> wrote:
>>
>> Hi, Lisa.  Thanks for the response, appreciate it.
>>
>> I can indeed download the sound file manually from the Making Contact
>> stations page, by right-clicking in my browser and selecting download.  But
>> when I try to retrieve that same file using the curl tool, I get this
>> (incomplete) HTML file instead of the MP3:
>>
>> <html><head>
>>
>> <META NAME="robots" CONTENT="noindex,nofollow">
>>
>> <script src="/_Incapsula_Resource?SWJIYLWA=5074 ... 932f3"></script>
>>
>> <body>
>>
>>
>> I'm open to anything the hive mind suggests.  <smile>
>>
>> Regards,
>> Mike.
>>
>>
>> On Fri, Jul 24, 2020 at 10:59 AM Making Contact <distrib at radioproject.org>
>> wrote:
>>
>>  Hi Mike
>>>  https://www.radioproject.org/sound/2020/MakingCon_200715_Ax.mp3   is an
>>>  MP3 from what i can see, and you download that
>>>
>>>  I don't know of an easy way to AUTOmate the weekly download if you are
>>>  accessing files from our stations page and not via pacifica's Audioport, or
>>>  PRSS etc.etc
>>>
>>>  I wonder if folks on the GRC list (cc'd here) might have a nifty trick?
>>>
>>>  Glad you are broadcasting Making Contact now let's see if the hive mind
>>>  can help you do so more easily!--Lisa
>>>
>>>  On Thu, Jul 23, 2020 at 6:24 AM Mike Carroll <mikec at kptz.org> wrote:
>>>
>>>  Gentlefolk:
>>>>
>>>>  I'm a volunteer with KPTZ, a community radio station in Port Townsend,
>>>>  Washington, and responsible for downloading the weekly Making Contact
>>>>  show.  I'm trying to automate this process using the Linux command "curl",
>>>>  but can't make that work.
>>>>
>>>>  For example, "curl
>>>>  https://www.radioproject.org/sound/2020/MakingCon_200715_Ax.mp3" returns
>>>>  a web page instead of the MP3 file.
>>>>
>>>>  Can you suggest a way to automate the weekly download?  Thanks for any
>>>>  insights.
>>>>
>>>>  Regards,
>>>>  Mike Carroll
>>>
>>>
>>>
>>>
>>>  --
>>>
>>>
>>>  <https://www.facebook.com/makingcontact/?eid=ARBPWLyeoMQs1yIayXMQ4_pPQsnJHq-TTUSbwAxYADoJzkPVup13455H_f6mnW5ACKxliIetiw0sck47>
>>>
>>>  <https://twitter.com/Making_Contact?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor>
>>>    <https://www.instagram.com/makingcontactradioproject/>
>>>  <https://soundcloud.com/makingcontact>
>>
>>
>> ------------------------------
>>
>> grc mailing list
>> grc at maillist.peak.org
>> http://maillist.peak.org/mailman/listinfo/grc
>>
>>