Data Aggregator
- Created by
- dMap Team
- data.australia.gov.au datasets used
Australia, World Heritage Areas
Murray-Darling Basin Boundary 2007
Commonwealth Marine Protected Areas Managed by DEWHA
Integrated Marine and Coastal Regionalisation of Australia (IMCRA) v4.0 – Provincial Bioregions
Local Government Boundaries Annual Extract Queensland
Australian Federal Electoral Boundaries
Location of legal assistance service providers
South Australian Recreational Boating Hazards Map
Australian Atlas of Mineral Resources, Mines and Processing Centres
Police Station Locations (VIC)
Public Internet Locations (VIC)
Detox Your Home Locations (VIC)
Economic Stimulus Package Project Data (community infrastructure & road & rail projects)
Northern Territory Place Names
- Other datasets used
ABS: Statistical Local Areas
ABS: Local Government Area
ABS: Indigenous Location
ABS: State Electoral Division
Geoscience Australia: Indigenous Land Use Agreements
Geoscience Australia: National Native Title Register
This mashup is for anyone who wants to use a single web application to explore multiple data.australia.gov.au datasets, to interactively combine datasets and to inspect data content without having to download all the original data files. From local community services to complex Geographic Information System (GIS) files.
Main Points:
- Combine different datasets on a single map (eg Indigenous Land Use Agreements + Mining Infrastructure + Operating Mines + Processing Centres + Shipping Ports).
- Identify sub-classifications of data (eg only display Internet Locations with Wheelchair access).
- Inspect data content from the original data files (eg view online the hazard photos from the Boating Hazards SA dataset).
- Verify data reliability (eg every mapped data item provides a pop-up window displaying information on when the data was last updated, the original data link, data owner and copyright details).
- Re-use the mashup. Anyone can add the mashup to their website to display open access data to their online community (eg the mashup is an embeddable application and can be easily added to web pages with a few lines of HTML code, similar to embedding Google Maps in a web page).
Data Transformation Challenge:
- Datasets used in this application are retrieved via AJAX/ JSON from our Data Directory service located on a separate URL.
- The Data Directory API provides a single interface and common data format to access data from different dataset formats (eg CSV, XLS, KML, GIS, etc).
- The Data Directory provides data transformations to enable complex and large GIS files to be mapped online with Google Maps. Transformations include Grouping, Smoothing and Google Polyline Encoding.
- The Data Directory can be re-used programmatically online by multiple applications and organisations via AJAX/ JSON.
Hi there,
This is an excellent product which can use spatial data well! However I’m not sure (like other entries) if this has or has not breached the rules:
You have clearly placed promotion of your product “dmaps” who provide consulting and products for mapping.
I guess the judges will really be the ones making this final decision.
Is there way to access the source code of this project – I would be interested in mapping some other data?
Thanks,
Jon
Glad you liked the mashup Jon.
Whilst we are a commercial organisation, we are also working hard to achieve what we believe will eventually become a significant and freely available community resource – this is what we have entered in the competition.
When you visited our website, perhaps you might have missed the section on the Fair Use Policy, as follows:
“At dMap, we believe open access to information and data ultimately underpins a successful and equitable society. We’re committed to initiatives that support this outcome. As such, public data sets in the dMapi (Data Directory) are freely provided under a Fair Use Policy. The main aim of the Fair Use Policy is to ensure that access and use of these online resources are within the community spirit. Fair Use means you must not do anything that impacts other people’s use of the dMap or dMapi services.”
The public data sets in this mashup are freely provided to us and we freely provide access to them via our Data Directory API under a Fair Use Policy. Also, this mashup is embeddable and can be freely re-used and embedded in most websites with the cut-and-paste of a few lines of HTML code. (Please note, this app has been put together for this competition and for demonstration purposes only, so should only be used for test purposes.)
In regards accessing the code – we are working on a public API for the embeddable mashup application which might be what you require but, that’s still under development.
Eventually, we hope not-for-profit groups will be able to simply do what you asked – to see their own data mapped. Then be able to aggregate their data with other public data sets, as well as collaboratively publishing their data content.
Hope that answers your question. Also, thanks again for your feedback on the app’s spatial capabilities.
Regards,
dMap Team
What I like about this app is a) the ability to overlay and compare just about all of the data sets on offer, and b) the way it handles resolution. eg. If I look at Economic Stimulus Package -> Project Type -> Community Infrastructure, I see regional/area counts where it makes sense to summarise plotted points (ie zoomed out), which then break apart as I zoom in (ie where I can make sense of the detail). Keeps the map clean and readable, and makes possible broad/regional comparisons. Nice work dMap.
(Disclaimer – I’ve watched at a distance as dMap put this together for mashupaustralia, and am now having a play).
Hey Simon,
Glad you liked it. Re your comments:
a) We aim to keep adding datasets from this and other open access data repositories – the value of these datasets will further increase when they can be easily combined/ aggregated with other data repositories.
b) As you say, the grouping keeps the map clean and readable.
You can also further configure the level of grouping to your specific requirements, as follows:
On the top right of the screen, click “Marker Defaults”.
Change the Marker options for “Marker Grouping” and “Marker Size”.
Also, with the “Proportional or Equal Size” option, the markers can be Equal size rather than the default Proportional size (where the area of a marker reflects the number of grouped addresses).
Thanks for the feedback.
Cheers,
Darrell
(NOTE: “Marker Defaults” wont impact markers you’ve already displayed, even after you hide and re-show them. To change defaults for all previously displayed markers, just click “Reset Map” then the new Marker Defaults will apply.)
dMap Team,
I like the categorical organization of information that can be seen on the map. I also like the marker aggregation at distant zoom levels. Would have liked to use this functionality in my own mashup but didn’t end up doing so after frustrations with the underlying service used!
Hi Harshit,
Browsing the entries the other day, I came across yours. I’m sure a lot of Victorians, including myself, will appreciate the social value of your work, also the practical value. Like the mashup too, looking forward to seeing the “Yet to come” features you’ve planned (Victoria: Fire Ready!).
Re your comments on the categorical organisation of data (see part “a” of my feedback above to Simon). The Data Directory menu structure allows us to add many more datasets. If the number of datasets starts to impact menu performance (no problems so far) we’ll switch to dynamically loading menus categories as the end-user drills down, also dynamic loading based on end-user entered search words. No reason we can’t have 1000’s of data sets in the app … if we can find that many.
Cheers,
Darrell – dMap Team
This application isn’t particularly intuitive, but wow is it thorough.
By covering pretty much all available data, this is a great companion tool for anyone building their own applications.
There are only two things I don’t like about this application.
1. The mapping applications seems to be a bit too aggressive doing polygon simplification, resulting in a lot of visual tearing.
2. What would make this an utterly essential tool would be if I could DOWNLOAD the data sets from this application in uniform formats. Factoring away all of the file format differences and providing a one-stop shop for boundary/point data would be just awesome.
But both of these complaints are minor. The fact I’m complaining about these things at all means that it’s gotten all the important things right.
Well done I say.
Hi Adam,
Appreciate the feedback and agree with your thoughts.
The 2 points you make – hopefully you’ll find the app already provides the functionality you require.
1) You can set the default level of polygon detail to your needs.
At the top right of the screen – click “Boundary Defaults”.
Change the “Accuracy” to “Highest”. This will then apply to all boundary items you subsequently load.
Also, when zooming down in the map, the detail will keep increasing all the way to street level with centimeter accuracy (depending on the original dataset).
(NOTE: “Marker Defaults” and “Boundary Defaults” wont impact content you’ve already displayed. To change previously displayed boundaries, click “Reset Map” then re-select the Data Directory items and the new Boundary Defaults will apply.)
Which highlights the point you make on the intuitiveness of the app. The app presents over 30 datasets, we could’ve presented them in separate more intuitive mashups – the framework supports this. Yet, the value of aggregating data would be lost. It’s a trade off, depending on the specific purpose and audience we would either produce a simplistic interface or one that can handle 100’s or more datasets.
2) Agree with your comment:
It does that, all the data sets, regardless of the original format (eg CSV, XLS, GIS, etc) or original data content, the Data Directory API serves the data in a uniform format.
Our front-end uses the same code and API methods regardless of how many more data sets we add or the format or content of the original data. The front-end never has to know what the original data format was.
The mashup can then display the original data content despite the differences in the original data sets. You can see this by drilling down to specific data items (boundaries or addresses), click on the item, it will display a Google Info Window, click the data link in the Info Window and a Data Window with the original data content should appear (for addresses you might need to zoom down further). The “Boating Hazards SA” is a good example, when it displays the original content, it also displays the hazard photo content.
We’re further developing the Data Directory API, when the API has stabilised we hope to make it an open community resource and other application developers can use it, as we do via x-domain AJAX/ JSON (based on the Fair Use comments in the post above).
Until then, the embedded app is also re-usable and can be easily inserted in other websites (please note: this mashup is just for demo or test purposes) – we also aim to provide an API for the embedded apps so that other organisations can choose the datasets to display that are relevant to their local online community.
As you mention,
Couldn’t agree more and this is what we aim to achieve with the Data Directory API. It’s early days but we hope this will eventually become a significant community resource – compounded by the benefits of centrally managing, maintaining and updating data.
Please feel welcome to voice your thoughts further – particularly if the above info doesn’t achieve what you have highlighted.
Thanks again for your feedback,
Darrell – dMap Team
Hopefully the above detailed response clarifies the features of this application.
For anyone wanting a more abridged version:
The mashup displays data from large complex GIS files of 100s Megs over the internet in seconds.
The level of detail is configurable and provides finely detailed polygon accuracy even at street level – similar to GIS software packages.
In regards the ideals expressed by Adam:
This is what the application does.
As more datasets from other open access repositories are added to the Data Directory API, even greater community benefit from these datasets will be derived.