Featured post

Welcome to ArchaeoPY

Welcome to ArchaeoPY!

We are a group based at the University of Bradford, UK, interested in using python for scientific applications. Our aim is foster a dynamic, open-source code repository through supportive learning and challenges. So if you're interested in coding or want to make your life easier when working with geospatial data or want to know your grd's from your csv's and dat's, hopefully ArchaeoPY will help.

If there is anything particular you would like a tutorial on, or have a specific idea in mind for a python application, leave us a comment!

And please check out our Getting Started Guide!

GitHub Inventory and Housekeeping

Hello ArchaeoPY,

We are thrilled by the positive responses to our paper at the 2015 CAA UK hosted at Bradford this past weekend.

Chrys and Finn have finished taking inventory of what is currently in the ArchaeoPY repository. As a result, we have shuffled some things around, renamed some code to be more descriptive, and deleted a few extraneous files. A short inventory of what is currently available below:

ArchaeoPY Modeller - 2D resistivity and magnetic responses modeller over simple features. 3D modelling has been derived, but still needs to be implemented. Hasn't been forgotten--just taken a back seat to other tasks!

ArchaeoPY:

Filters: Contains filters (analytical signal and peak-finding, which for example, we recently used on EMI and mag data)

GUI Templates: Contains the base ArchaeoPY templates generated from Qt Designer... These may be re-thought at a later time. Originally ArchaeoPY was meant to have a template GUI, to which users could utilise to their own needs by importing in modules. Currently this template GUI is mpl, which contains a matplotlib canvas, with plotting and clearing buttons.

IO: Various in>out code. Includes binary data viewer and binary data savers (to dxf, txt, or png); image processing and saving GUI; EMI regridding GUI.

Plotting: Basic plotter GUI, now with statistical information and trendline plotting; multiple subplots plotting from the same file; and, in progress, multiple subplots from different files.

Positional: EMI (xyz data) regridding, grid sorting, geotiff with open layers.

Processing: scikit image processing, geophysical data processing.

We are still getting the hang of GitHub; so if you have any questions or comments, please leave them below or tweet/facebook/google groups us!

 

 

ArchaeoPY Update: What's new... What's to Come...

Hi All,

It has been awhile since our last update, but we are still alive and active!

Our GitHub repository  is slowly growing... so much so that we are in the process of doing an "inventory," to better structure the organisation of the repository. End goal: restructure snippets of codes (e.g. filters, data loaders) into folders separate from the GUI's, and have template GUI's, which the user can import and utilise the various snippets to suit their needs.

We will try to provide more regular updates on what we are working on through this page, but the GitHub repository will always be the most up-to-date. Currently we are working on:

Plotting: We have developed a straightforward 2D plotting interface (plotter) that loads tab separated data. This has developed out a need to plot lines of EMI data. The end goal is to add multi-axis plotting capabilities and run basic statistical analysis of the data. The GUI needs tidying up, but it still does the job.

plotter

An extension of Plotter is the multiple subplots GUI, which is still in progress:  multiple subplots

Other things we are working on also include a binary data viewer > exporter, and lidar hillshade data processing. Stay tuned for more.

 

ArchaeoPY: Constructing and Utilising Open Source Software for Archaeological Geophysics

This Presentation was given at the "Recent Work in Archaeological Geophysics" conference In December 2014.

ArchaeoPY meetings

Welcome new ArchaeoPY members! Please find the online ArchaeoPY presentation here.  With the beginning of the new University term, ArchaeoPY will be more active, with regular meetings scheduled every Thursday, from 12:00 - 1:00 PM in the Master's computer room, Phoenix SW. For non-Bradford based members, the meetings' topics, information, and any discussions arising from the meeting, will be posted here. If there are members outside of Bradford who are interested in being involved, we can work out a way of streaming the meetings.

Exciting new changes are being implemented, with ArchaeoPY switching to an iPython notebook based server: more info, coming soon on ArchaeoPY.org. In the mean time, follow us on Twitter, like us on Facebook, or subscribe to the ArchaeoPY Google Groups to receive notifications.

Tutorial 2: Numpy Arrays

Hello Archaeopy! We will be releasing our Geophysical challenge two next week! To get you ready for this challenge, here is the complementary tutorial covering numpy arrays. We have discussed numpy before in previous posts, but since it is the fundamental package for handling our data, we thought it would be important to cover the numpy basics once more. Much of this tutorial will be taken from the tutorial on the official numpy webpage. Check it out for further information.

Why numpy? 

Numpy's main object is the multidimensional array. Consider geophysical data: we have positional coordinates (x and y) and at least one value (z). There are many ways we can handle this information (e.g. xyz file, grd file, as profiles). Numpy arrays are efficient for handling, storing, and manipulating large datasets.

Continue reading

My first Py

Recently I went out on site and collected two reasonably large data sets which could not be easily handled by proprietary software.  This rather fundamental need to be able to actually use my survey data inspired me to have a go at the first ArchaeoPy challenge and to start to get to grips with the Python language.  Having completed and uploaded some script, I was encouraged by Popefinn and Chrys Harris to share it with you in this post.

Continue reading

Terminology

Hello Archaeopy! It has been awhile since our last post. It has been a busy past two months full of conferences and fieldwork, but these have inspired upcoming tutorials and challenges! We would appreciate some feedback regarding how ArchaeoPy is administrated. Our new plan is to release challenges monthly (with the next challenge issued next week, on the 17th... so there is still time to complete the first challenge and submit your entry for the mystery prize!) with virtual meetings being held and broadcast every fortnight. Also, what do people think of a mailing list, for the between meeting times, for any questions, comments, or ideas? We will we be sending around invitations to a google group, to those who are subscribed as archaeopy.org users. Please let us know in the comments below, on the facebook page, or via tweet what you think of these changes!

Continue reading

Geophysical Challenge 1 - Need some help?

It has been a few weeks since we have posted our first geophysical challenge. We thought to create a tutorial post for the challenge to help you along! First things first, you will need to download the data for this challenge. Save this data to the same folder you will be saving your python challenge 1 script to. Now open Spyder. We have provided a step-by-step tutorial with information about each step, to help you along.

Continue reading

Behind the Curtain: Modules, Packages, and Libraries

Following a question asked after our Geolunch session, "What is numpy?" We thought it would be useful to provide a brief overview of python's modules, packages, and libraries: what they are and how they are useful.

Since ArchaeoPy focuses on scientific applications, we often want to reuse the same functions and variables. Luckily for us, python has a great selection of modules and packages that we can use to simplify our projects; instead of having to define our own functions and variables with each project.

A module is a file that contains definitions of functions, variables, and classes that can be imported and run in other modules. A package is a directory of modules. A library is a collection of packages and modules.

Python already has a great standard library with built-in functions, constants, and types that we will regularly use. However, we will often want and need to go beyond the standard library tools, and to that end, there is a range of other powerful numeric and scientific libraries and modules that we can utilise. Below lists just few of which we frequently utilise for our scientific applications (hopefully familiar from the code in our Geolunch presentation and in our ArchaeoPy repository):

NumPy: Numpy is one of our fundamental packages because it handles and processes arrays, which are what we use to structure our geophysical data.

matplotlib: matplotlib is a plotting library. In our examples, we have used matplotlib to produce greyscales.

PyQt: We have used PyQt to create our graphical user interfaces (GUI).

Geophysical Challenge 1

Following on from yesterday's Geolunch, here's the first Applied Geophysical ArchaeoPy challenge:

Load XYZ .CSV file and plot the data within python

You can do this in any way you want but some hints and tips are given below:

  1. Download the data from http://www.archaeopy.org/wp-content/uploads/2014/04/xyz.csv
  2. Numpy.loadtxt to load in a CSV file
  3. Matplotlib.mlab.griddata to grid the XYZ data
  4. Matplotlib.pyplot to plot the data
  5. If you get stuck, google is your friend!
Extra Credit
Its not very often that we use completely raw geophysical data. Usually we'll have to apply some processing steps to make the data more usable.
A simple initial processing step is a Zero Mean Traverse. We don't need any extra software packages to do this.
Numpy.mean can be used to calculate the mean of a group of values, or more usefully return the means of rows/columns in an array.
Numpy.subtract can be used to remove that value from rows/columns.
You might want to use loops and array indexing to do this but you shouldn't have to.
Upload your example to the ArchaeoPY Bitbucket (details here).
Prize for the most readable and well documented code.