The next part on the series includes figuring out if a given state park actually has camping enabled at this time. This should be an interesting return of data because COVID-19 has made it not consistent across all parks (this could change in the coming weeks as cases go up).
As it works currently we have a
getStateParks function which returns an
IO [(Key StatePark, StatePark)]. Currently a given state park will always have
hasReservation set to false. Let’s change that and make that Boolean live (as in query the web about it).
Looking through the pages we can see that a given page will have a specific link if state park has reservations enabled. Now we need a function that checks for this link. In the form of a scraper its as easy as
reservationStatusScraper :: Scraper Text [Text] reservationStatusScraper = attrs "href" "a"
With the above we can then use
Prelude.elem to check if the link exists on the page.
At this point in time I realized that I can also change my data model references from
Text to help simplify constructing the
StatePark model. This helped to simplify my
slugify :: Text -> Text slugify s = let repl ' ' = '-' repl c = c in toLower $ (Data.Text.map repl (s))
Continuing on we can compose another function in
getStateParks to do the data transformation of updating
getReservationStatus will be applied on every link from the
ScrapedStatePark so that means we will have a type signature kind of like
getReservationStatus :: ScrapedStatePark -> IO (Key StatePark, StatePark)
In order to compose these functions we can use an
IO Monad function for executing a sequence of
IO operations called
sequence. Combining them all looks like the following.
getStateParks :: IO [(Key StatePark, StatePark)] getStateParks = do scrapedFromURL <- scrapeURL stateParksURL stateParksScraper sequence (Prelude.map (\r -> getReservationStatus r) (fromJust scrapedFromURL)) getReservationStatus :: ScrapedStatePark -> IO (Key StatePark, StatePark) getReservationStatus (ScrapedStatePark n l _hasReservation) = do linksOnPage <- scrapeURL (unpack l) reservationStatusScraper return $ (StateParkKey (slugify n) , StatePark (n) (slugify n) (l) (Prelude.elem ("http://www.reservecalifornia.com/" :: Text) (fromJust linksOnPage)))
Its kind of cool to think that we are working on the Monad level with
sequence because we are moving the
 angle brackets inward one level. This is so you can actually process the inner data since it starts off as a series of separate
[IO (Key StatePark, StatePark)] -> IO [(Key StatePark, StatePark)]
After gluing the functions together I now realize that its kind of slow running. That is why in my next article I’m going to make the pull for status parallel instead of a sequence.
Progress is on GitHub.