First Demonstration of ECHO: an External Calibrator for Hydrogen Observatories

Beam calibration of low-frequency dipole arrays poses several complications compared to traditional dish antennas. Most notable is that traditional beam calibration, where the beam is scanned across a bright isolated source, is not possible because arrays are fixed to the earth. Phased array beams are technically steerable but violate the assumption that the beam is constant as it scans.

Drift scan calibration of dipole array beams is possible—observing point sources as they track across the beam—but it requires the test antenna/array to be embedded within an existing array that generates sufficient sensitivity to isolate a large number of radio sources to provide many tracks through the beam or availability of a large steerable dish with which to cross-correlate Berger et al. (2016). For dipole arrays with large beams, drift scan calibration also requires stable instrument responses over hours, which may be difficult to achieve due to ambient temperature variations. Even if source tracks can be measured at high sensitivity, there is still the issue that sky sources only measure right-ascension tracks along constant declination and cannot as easily constrain the beam in the orthogonal directional. Pober et al. (2013) have employed symmetry arguments to tie different right-ascension tracks together, but this is not possible in general for more complicated dipole array beam patterns and is ultimately limited by the symmetry argument and in the sensitivity far out in the beam.

Attempts have been made to use anechoic chambers to calibrate low-frequency phased arrays, but the measurements suffer since even the largest chambers cannot extend into the farfield pattern and the RF absorber material used in the chambers performs poorly below 150 MHz, creating reflections and resonances and, in any case, does not capture the as-built variation. Mapping beam responses of antennas in the field using the Orbcomm constellation of satellites has proven the most effective (Neben et al. 2015, 2016a) but is limited to only a single frequency (138 MHz).

Drone-based radio calibration has recently been explored for application to widefield 21 cm instruments. A drone-mounted calibration source was used to verify the accuracy of antenna response modeling for SKA-low stations Virone et al. (2014) and then a second generation setup was then used as a phase calibrator (Pupillo et al. 2015). Fully mapping the beam of an antenna has been demonstrated by Chang et al. (2015). In that experiment, a 5 m dish was mapped at 1 GHz with a noise source broadcast by a gimbal-mounted horn, providing a first demonstration of the drone beam-mapping concept and identifing places for improvement of the methods. Future improvement areas they identified included: optimizing the flight path to better sample the beam, obtaining better drone positioning accuracy, improving characterization of the transmitter beam, and better modeling of the antenna under test (AUT). This last point was partially motivated by a lack of alternate beam mapping data with which to compare the results.

Changes from the Chang et al. (2015) system must be made to shift from 1 GHz to lower frequencies used for reionization experiments. Below ∼1 GHz, horns and other highly directive elements become prohibitively large and heavy for flight applications, motivating the use of electically compact antennas, such as dipoles like those used by Virone et al. (2014). Fixing the transmitter antenna to the drone without a gimbal further reduces structural complexity and weight, but increases the need to understand the transmitter beam pattern since the orientation of the transmitting antenna will vary relative to the AUT.

Drone and other airborne calibrators are particularly well-suited to instruments with a reachable far-field distance. The far field distance is defined as the distance at which wavefronts are planar across the element in question, an approximate estimate for the far field distance is the square of the aperture size over the wavelength. Table 1 lists several representative 21 cm arrays and their approximate far field distances. The far field for HERA is ∼100 m, which is a reasonable operating height for a small drone. While desirable, we note that the far field threshold is flexible, as techniques exist to transform near field and Fresnel zone measurements into the far field (Johnson et al. 1973).

In this paper, we report our progress toward overcoming the challenges of drone-based calibration with the development of the External Calibrator for Hydrogen Observatories (ECHO), which specifically addresses the 21 cm requirements identified above. The ECHO system is described in detail in Section 2. It uses a broadband dipole as a transmitting antenna mounted in a fixed configuration beneath a multi-rotor drone. The antenna is driven by a sine-wave source that provides high transmission power at a tunable frequency. Beam maps are captured by flying the transmitter over the antenna using the HEALpix pixelation scheme to establish sampling waypoints. As a test of the method, we have mapped two reference dipoles utilized in the beam-pattern measurement system of Neben et al. (2016a) that uses the Orbcomm low-Earth orbit satellite constellation as transmission sources to map the difference between two identical antennas. Though the Orbcomm data collected simultaneously provides a reference point for the stability of the drone it does not provide a direct measurement of the beam. For a direct check on our beam maps we turn to simulations. Using a combination of these two we quantify the measurement stability and overall accuracy of the ECHO system. We discuss the results of our analysis in Section 3, and in Section 4 we summarize the conclusions from these tests and identify places for further improvement.

We use an X8 octoquad from 3DRobotics (3DR) with a Pixhawk autopilot running Arducopter 3.5. With a 10,000 mAh battery and a 1 kg payload, this setup has a flight time of ∼15 minutes. Position is recorded by a Ublox GPS and a barometric altimeter, which together achieve a sub-meter position error. The transmitter is a programmable Valon 5007 voltage controlled oscillator. Typically used as a clock signal, this device can produce a stable, continuous-wave signal tunable from 137 MHz to 5 GHz and is programmable via USB interface. A continuous-wave signal is chosen over a broadband signal for our first testing because the power is concentrated in a narrow band and does not require additional amplification. This reduces both weight and complexity and enables better flexibility to avoid interference. In the work reported here, the transmitter is programmed to broadcast at 137.5 MHz with attenuation added to produce −22 dBm of output power. This output power is chosen to reach a peak SNR of 40 dB in our receiving station without evidence of saturation. The transmit antenna is a BicoLOG 30100 biconical antenna manufactured by Aaronia. It is passive, has a smooth spectral response over an operational spectral range of 20 MHz to 3 GHz, and is light-weight yet robust.

The antenna mounting is chosen to maximize the distance from the drone to facilitate modeling of the transmitting beam pattern. The X8 drone legs are too short to permit a fixed mounting further than two or three centimeters below the drone and the wingspan is too narrow for safe landing on longer legs. In our tests, the antenna has been suspended from the drone legs by non-conductive monofilament with a plastic bracket providing the filament/antenna mount point. On the ground, the antenna fits between the legs, in the air the antenna hangs 30 cm below, as shown in Figure 1. The advantages of this system are that it is simple, extremely light-weight and provides isolation between drone and antenna. The disadvantages are that it lacks rigidity, stability, and because it must be re-tied after transport it is difficult to repeat precise alignment between tests, requiring additional beam modeling. In this scheme, with the antenna fixed in position relative to the drone, the drone yaw (rotation about the vertical) controls the transmitter polarization angle.

Zoom In Zoom Out Reset image size

One of the main difficulties encountered by Chang et al. (2015) was in matching the flight pattern to the size scale of the variation in the beam. They chose a cartesian grid pattern with a separation such that the drone passed through the primary lobe roughly three times. When combined with the on/off switching they used to subtract the 1 GHz noise of the drone motors, the number density of samples became extremely limited and led to spurious results when interpolating to make a complete beam map. Here, our goal is to smoothly map the beam of a very wide-field element from zenith to the horizon, which would be difficult with a constant altitude flight pattern. This is also true of the analysis pipelines where the distortion caused by the gnomic projection to a tangent plane makes for a poor gridding scheme at the horizon.

In many 21 cm pipelines, the HEALPix pixelization scheme (Górski et al. 2005) is used to store images and beam models. HEALPix divides the sky into equal area pixels with the pixel count and resolution selectable by powers of two. Pixels can be ordered either by their position in the hierarchical doubling (NEST) or in a spiral order with longitude ϕ fastest (RING). We chose a flight pattern which follows the RING pattern to give us a spherical shell flight pattern centered on the receiving antenna with a chosen radius. Following a conservative safety practice limiting our drone flights to within a good line of sight of the operator, and considering the FAA flight ceiling of 400 feet, we set the radius (and thus the maximum altitude) to 100 m (328 feet). The NSIDE parameter sets the number of waypoints and, effectively, the resolution of the beam sampling. This number is constrained, ultimately, by the amount of time available to create a beam map—at constant speed, a denser grid will take longer to record. The slowest variable to be sampled in our telemetry is GPS position at 2 Hz. We've chosen to fly at 1 m/s, hence the drone will acquire about three samples per angular degree with a HEALpix NSIDE of 8. At this resolution and speed, the full $2\pi$ steradians of a beam response above the horizon can be mapped to a resolution of 9° in 60 minutes with four 15 minute sorties. Figure 2 illustrates the ECHO coordinate system and flight path.

Zoom In Zoom Out Reset image size

The polarization angle of the transmitting antenna is kept fixed with respect to the drone, the drone is kept at a fixed orientation by inserting a Region of Interest (ROI) command into the beginning of the waypoint program. The ROI command causes the drone to maintain a fixed heading in the direction of a set location which we choose to be a distant location due East or North to match the polarization of the antenna being mapped.

We tested ECHO by mapping two antennas that are used in the Orbcomm-based beam mapping system developed by Neben et al. (2015, 2016a) at Green Bank Observatory.⁵ The Orbcomm system measures the power received from multiple satellites, operating in narrow band channels near 137 MHz, taking advantage of the telemetry in Orbcomm signals to associate the received channel powers with the orbital positions of the satellites. The time variation of the digital signal is removed by measuring the ratio of power received by the AUT and a nearby reference antenna.

The reference antenna is a dual polarization, sleeved dipole over a 2 m × 2 m wire mesh ground plane shown in Figure 3. This antenna acts as a precision calibration element and can be calibrated out using a simple beam model. The accuracy and repeatability of this scheme is established with a "null test" where the AUT is replaced with a second identical reference dipole. Comparing these nominally identical antennas gives an estimate of systematic errors caused by noise, multi-path, or antenna construction.

Zoom In Zoom Out Reset image size

The Orbcomm system was in this "null test" configuration, shown in Figure 3, in preparation for mapping the HERA beam (Neben et al. 2016a) when we performed the ECHO tests described here. We used both of the system's two reference dipoles as our AUTs for ECHO. This will let us compare the individual ECHO maps with the EM model and compare the differenced maps with the measured Orbcomm ratios. In the rest of this paper, we will refer to the two AUTs as antenna A and B, respectively, and the two dipole polarizations of each antenna as east–west (EW) and north–south (NS).

Received power by our AUTs was collected using the spectrometer setup of the Orbcomm satellite mapping array as described in Neben et al. (2016a), their Section 2.4. The software was modified to increase the spectral dump rate and allow real-time inspection of the data, but otherwise the system was operated just as it was for Orbcomm-based beam mapping, with spectra recorded from our two AUTs at a time cadence of 4 Hz and a spectral resolution of 2 kHz. Although spectra were recorded simultaneously from both AUTs, we made separate flights centered on each AUT individually for the measurements reported here. We only analyze data from an AUT at the center of a flight pattern.

We generate beam maps by interpolating the received power trace at times when recorded GPS positions are available. Careful observation of the autopilot behavior and inspection of logs suggests that the X8/Arducopter combination suffers from occasional stability "hiccups." Finding that these are characterized primarily by outliers in the yaw, we have flagged points greater than 2σ above the mean yaw. Another slight yaw outlier is observed to occur as the drone passes through each waypoint. These are often accompanied by visible bumps or dips in the received power trace, as can be seen in Figure 4 near each of the vertical lines marking a waypoint passage. Points within half a second of each waypoint have been flagged.

Zoom In Zoom Out Reset image size

After flagging, data are gridded using a nearest pixel approach into a HEALpix grid using an NSIDE of 8, the same pixel scheme as used to create the flight pattern. The gridder records the mean, standard deviation, and total sample count of all the samples falling into each pixel. An example gridded pattern is shown in Figure 5. As is typical for systems dominated by multiplicative errors, we observe that variation within pixels has a log-normal distribution and so record the standard deviation of the log. In some cases the number of data points becomes small which increases the uncertainty on the standard deviation. Fractional error in the standard deviation goes as $f=1/\sqrt{(}2(N-1))$. To make the most conservative estimate of our error bars we report here the upper bound of this uncertainty; the standard deviation multiplied by this fractional increase. Small number counts affect about 20% of our map pixels with an increase of at least 0.1 dB.

Zoom In Zoom Out Reset image size

The measured power pattern is a product of the power patterns of the transmitter beam and receiving AUT beam. To remove the effect of the transmitter beam, we divide out by a simulation of the transmitter beam created in CST Microwave Studio based on a mechanical model of the drone plus antenna. The model is shown in Figure 6 and includes the biconical transmitter antenna, as well as the metal arms of the drone. The central body of the drone (shown as a translucent gray rectangle) is made of plastic and carbon fiber and contains control electronics, batteries, and the transmitter circuitry. These components are not modeled, nor were the motors on the ends of the arms.

Zoom In Zoom Out Reset image size

In Figure 5, we can see the basic features of a dipole with its polarization axis oriented north–south above a ground plane: a nearly uniform response along the east–west direction and deep nulls to the north and south. In the nulls beyond ∼70° from the zenith, the amplitude falls more than 30 dB below the peak response. The standard deviation between measurements within a gridded pixel is largest near the nulls, reaching 2 dB, compared to ≲0.4 dB in the main part of the beam. We take this sampling noise as a reasonable estimate of the statistical uncertainty in the discussion below.

We compare the measured beam patterns against an electromagnetic model in order to examine each of the four measurements independently and assess the accuracy of the measurements. As sleeved dipoles above small ground planes, antennas A and B are relatively straightforward to simulate in modern codes such as CST, FEKO, and HFSS. Here we use the CST model constructed by Neben et al. (2015) for use in the Orbcomm experiment. The model includes one active polarization with the orthogonal polarization as a passive element, the ground screen, and a meter margin of surrounding earth. 90° symmetry is assumed for the second polarization. In Figure 7, we plot E and H-plane cross-sections through the beam patterns measured by ECHO, as well as cross-sections for the simulated beams for the AUTs. The cross-sections cut through the maps along the E-plane (parallel to the receiving dipole) in the top panels, and the H-plane (perpendicular to the receiving dipole) in the bottom panels. The E-plane passes through the the dipole nulls, while the H-plane passes through the part of the beam with the flattest response. The right column shows the cross-sections for the EW polarization of the AUTs. In this column, we see that the beams of the two AUTs agree to within the empirical sample noise errorbars in most cases. Agreement with the model is also within the sample noise errorbars. The antenna B measurements were peformed to lower elevations than those for antenna A and we see that the agreement with the model is good even for the lowest elevation points with antenna B, which reached $\theta =\pm 80$° and a received power 25 dB below the peak.

Zoom In Zoom Out Reset image size

The left column of Figure 7 shows cross-sections from the beam measurements of the NS polarization of the AUTs. We note that the measured beam pattern for the NS polarization of antenna A has been corrected for a misalignment of the transmitting antenna, which we will discuss in detail below. Measurements of antenna B were unaffected. Here, we can see that antennas A and B agree well with each other, even at received power levels of 30 dB below peak response. However, neither antenna agrees well with the model in the NS polarization. There is a systematic offset between both AUTs and the electromagnetic model, particularly in the E-plane for $\theta \lt 0$°. The NS polarization cases have deeper nulls in the E-plane and a flatter response along the H-plane than the EW cross-sections. Due to the symmetry of the AUTs, we would expect the EW and NS polarizations to yield identical beam cross-sections in their respective E and H-planes since we always align our transmitter antenna to the active dipole axis. Hence, it is surprising to see any discrepancy between polarizations. Since both antennas exhibit the same behavior, it suggests the issue is with the antennas themselves, and is not stability issue with the ECHO system or a systematic error in the modeled transmitter beam pattern. We investigated if a simple rotation of the NS arms relative to the EW arms within the AUTs can account for this difference and found that it cannot. Another possibility is that an unmodeled difference in the construction of the NS dipoles compared to the EW may cause the beam model to be inaccurate. Small geometrical asymmetries near the connection between a dipole and its active balun are known to cause beam distortions and we speculate that something of this nature may be the case in the NS polarization dipoles.

Most beam maps agree well with the model to within sample variance, however not all. One possibility could be systematic errors causing some maps to be better than others. Therefor we also compared the difference between antennas to the difference measured by the Orbcomm system.

We took the ratio between the ECHO maps for antennas A and B, treating the NS and EW dipole polarizations separately. Any instability in ECHO, differences between the two antennas due to construction irregularities or nearby environmental properties will result in non-unity ratios. The latter two systematics are captured by simultaneous measurements of the Orbcomm satellites.

Comparing the ECHO and Orbcomm-derived beam ratios, we found that the beam patterns measured by ECHO for the EW dipoles yielded ratios similar to those derived from the Orbcomm measurements, but the NS dipoles displayed a systematic slope in the NS direction, as evident in Figure 8. A systematic slope in the measured beam ratio of the two AUTs that is aligned with the polarization of the transmitter can be produced by a modest tilt in the transmitting antenna or one of the receiving antennas. Since the Orbcomm data do not show a similar feature, we assume the slope is due to a tilt in the ECHO transmitting antenna. Such a tilt is a likely failure mode of the monofilament transmitter suspension method.

Zoom In Zoom Out Reset image size

If stable within any given mapping run, the tilt can be accounted for during our data reduction by modifying the transmitter beam model used to normalize the data. Inspection of photos taken during the field trials revealed several instances of small antenna mounting tilts, mostly in the last mapping run of the experiment, on the NS polarization of station A. We find that a rotation of the transmitter antenna by ${\rm{\Delta }}\theta =10$° fits the observed slope well while staying within the bounds of likely misalignments observed in the field and in the photos. To compensate for the slope, we have applied a revised transmitter beam model that accounts for the misalignment of the transmitting antenna during the ECHO measurements of the NS dipole of antenna A. An unrotated transmission beam is used to normalize the other beam measurements.

The ratios between the measured antenna beam patterns are plotted in Figures 9 and 10. Comparing Figure 8 to the beam-tilt corrected version plotted in the middle panel of Figure 9, we can see that the original 10 dB slope across the beam has been reduced to a much smaller difference of ∼0.5 dB, which is similar to the typical sample noise in the ratio maps, as shown in the bottom panel of Figure 9.

Zoom In Zoom Out Reset image size

The ratio maps show again that ECHO measurements of antennas A and B exhibit nearly identical beam patterns to within the sample noise, except at large zenith angles. This discrepancy between the beam patterns of antennas A and B at large $| \theta |$ is especially evident in the EW polarization. At large zenith angles, the ECHO drone is close to the ground. It is likely that ground propagation effects influence the transmitted signal at large $| \theta |$, but are not accounted for in our analysis presently. Overall, the Orbcomm and ECHO data agree that the NS polarizations are not as similar to each other as the EW polarizations for the two antennas.

Figure 10 shows that the typical sample noise in the ratio maps is less than 1 dB, but climbs to 3 dB at the beam nulls. Any deviation between the ratios derived from the ECHO and Orbcomm measurements is typically within this level of measurment variation. Taken in conjunction with the agreement between the ECHO beam patterns and the simulated AUT beam patterns, the agreement between the ECHO ratio maps and the Orbcomm ratio maps suggests that the ECHO system yields accurate measurements of the AUTs. In percentage terms, the lowest sample noise is in the H-plane, where the deviations between samples within a given pixel were typically 1%. These variations are roughly on the same scale as the measured difference between the two antennas, as well as the difference between modeled beam patterns and data.

Zoom In Zoom Out Reset image size

Sample noise increases approximately linearly with azimuthal angle and reaches a maximum of 6% in the E-plane at high zenith angles, where the AUT and transmitter dipoles have their nulls aligned. Around the nulls, small rotations of the transmitter antenna can cause large changes in the received power. If these rotations are not accounted for in the drone orientation telemetry, they will yield errors in the beam patterns. Applying a Gaussian random rotation to the transmitter beam model, we find that the observed fractional variation near the nulls is consistent with a transmitter polarization rotation rms of 13. This is not far from the observed rms of the drone yaw of 22. The measured yaw is likely to be influenced by other errors such as magnetic interference from the motor currents and is likely an overestimate of actual drone motion.

The beam mapping experiment by Chang et al. (2015) used a rectilinear grid and found difficulty efficiently sampling the beam with a uniform coverage in a reasonable amount of time. In that case, the AUT was a dish with a fairly narrow field of view coverable with a level flight path. A level rectinilear grid becomes even more problematic when mapping a wide-field response. Measuring the response to the horizon would require a very large flight pattern. Given the requirement of sampling the beam over the full half sphere we chose as our flight path the ring pattern HEALpix pixelization. This is a signficant advance over regular level grids as it makes possible sampling near to the horizon level without having to fly far away. However, it does little to ease the time required to acquire high angular resolution maps, which require dense flight patterns regardless of gridding scheme. Nor does it necessarily reduce the sample variance, which would require slower flight speeds to obtain more samples per gridded pixel. Both issues strongly motivate the need for longer flight times. With a flight lifetime of 15 minutes for ECHO, each map reported here required four flights, which when performed with no technical issues, took about 100 minutes of total operational time since the downtime between flights to swap batteries adds an overhead of nearly 70%. A drone with a 30 minute flight time is possible with currently available technology and would make possible higher resolution maps in the same amount of operational time.

We found that under most circumstances, the repeatability of multiple measurements in a gridded pixel had a typical standard deviation of 1% after flagging on the telemetry data. However we see that the standard deviation increases as transmitter polarization becomes aligned with the receiver near the nulls of the beam patterns. In the null of the AUT beam, the standard deviation was about 2 dB or ∼60%. One possible source of this instability is if the transmitter antenna were rotating in its harness. Turning to our model of the transmitter beam, we found a yaw rms of 13 is sufficient to cause a 2 dB of variation. This variation is likely to be a combination of actual yaw instability in the drone and instability of the monofilament mounting method.

The sling mounting was chosen under the assumption that distance from the drone would improve the accuracy of the transmitter beam model, however subsequent modeling in CST has shown that changes in mounting distance yield small changes to the beam. Given the instability of the sling method, investigation is currently under way into options for solid mounts closer to the drone body. Furthermore, since measurement variance is consistent with excessive yaw in the system, we must ensure more stable flight and antenna mounting. Care must be taken to shield the drone magnetometer from the high currents feeding the prop motors and to properly tune the flight control loop settings to yield slow and stable turning. Finally, due to increased inertia, stability increases with the size of the drone, hence larger drones may be better suited for the application than the one used in this work.

In the end, drone stability can only be refined so far. In most drone applications, unwanted motion is typically further suppressed with a self-stabilizing gimbal which is commonly used for photography and cinematography. Use of such a thing was avoided in this first iteration to minimize weight and to avoid the addition of more conductive components. As this instability is the largest source of error, the benefits from addition of a gimbal mechanism will likely outweigh the costs.

Comparing our data with the electromagnetic model of the receiver dipole, we found that our data agreed with the model on one of the polarizations, but not the other. This was true for both identically constructed test stations. As these stations serve as the underlying basis for the accuracy of both drone and Orbcomm systems, future work should focus on identifying the sources of these differences and improving the accuracy of the models.

Other areas for further development include extension across the 21 cm bands. Coverage of the reionization (100–200 MHz) and BAO (400–800 MHz) bands is possible with the current configuration. A larger antenna will be required for the cosmic dawn band 50–100 MHz. ECHO measurements are suitable to characterize the polarization characteristics of an AUT and we plan to explore cross-polarization measurements in future work. The ECHO system suffers from sensitivity issues at the poles due to our restriction that the transmitting dipole antenna is maintained parallel with the ground. Transmitter feeds that transmit both polarizations simultaneously and are gimbaled to point at the AUT are possible, particularly in the sub-meter bands, however more investigation is required to understand the limitations and tradeoffs.

Is the quality of the mapping at the level required for precision foreground subtraction? If we take as a worst-case, the foreground precision requirement to be 10,000:1, then our target precision is 0.01%, while our current best demonstrated absolute accuracy is ∼1% but gets to 100% in the sidelobes. An error propagation study by Ewall-Wice et al. (2016b) suggests that a flat 1% (0.04 dB) precision of the HERA beam provides sufficient suppression of foreground residuals to enable detection of most power spectrum modes outside the wedge. In a similar study, Shaw et al. (2015) have set a requirement to know the CHIME to beamwidth to 0.1%, which for a Gaussian beam translates to ∼1 dB error bars at a −40 dB level. Here we have demonstrated 2 dB at the −30 dB level. More work is required to refine the target accuracy, as well as the best way to deliver and verify that accuracy, however it does appear to be achievable with modest performance improvements to the current system.